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INTRODUCTION  TO  THE  PROCEEDINGS  FOR  THE 
THIRD  ANNUAL  SYMPOSIUM  ON  SITU  I TIONAL  A  WARENESS 
IN  THE  TACTICAL  AIR  ENVIRONMENT 


Background 

The  3rd  Annual  Symposium  on  Situational  Awareness  (SA)  in  the  Tactical  Air  Environment  was 
held  on  2  and  3  June  1998  at  the  Paul  Hall  Center  in  Piney  Point,  Maryland.  The  symposium  was 
sponsored  by  the  Electronic  Warfare  Advanced  Technology  Program,  Naval  Air  Systems 
Command  (PMA-272).  The  symposium  was  coordinated  and  hosted  by  the  SA  Integrated  Product 
Team  (IPT)  at  Patuxent  Ri\cr;  points  of  contact:  LT  Meghan  Carmody-Bubb  at  301-342-9265, 
Karen  Garner  at  301-342-2985,  and  Tom  Assenmacher  at  301-342-0026. 

Purpose 

The  objective  of  the  symposium  was  to  provide  program  managers,  system  developers,  and  system 
users  with  a  heightened  appreciation  for  potential  SA  improvements  in  tactical  aviation  through 
the  focus  areas  for  the  1998  symposium:  Cognitive  /  Intuitive  Interfaces,  Multi-modal  Interfaces, 
Spatial  Awareness  Interface  Considerations,  and  Validated  Situational  Awareness  Performance 
Measures. 

The  symposium  provided  a  unique  opportunity  to  discuss  how  SA  influences  design;  learn  new 
ways  to  research  SA  in  the  tactical  air  environment;  learn  the  latest  developments  in  SA-related 
technologies;  discuss  SA  with  experts  on  panels  and  on  a  one-to-one  basis;  and  network  with  a 
variety  of  SA  researches  from  government,  industry’,  and  academia. 

Description  of  Proceedings 

Twenty-nine  presentations  were  given  during  the  symposium.  This  document  contains  formal 
papers  based  on  those  presentations.  Where  papers  are  not  available,  executive  summaries 
previously  printed  in  the  symposium  notebook  are  reprinted  in  the  proceedings  to  provide 
comprehensive  documentation  of  topics  and  authors  for  your  reference. 

Personal  Note 

Even  with  today’s  highly  accurate  and  effective  weapons,  tactical  airborne  mission  effectiveness 
depends  on  the  aircrew  achieving  and  maintaining  a  high  level  of  situational  awareness  throughout 
the  entire  mission.  This  can  be  done  first  by  recognizing  the  capabilities  and  limitations  of  the 
human  operator  and  designing  /  upgrading  systems  based  on  these  faetors. 

I  hope  that  those  who  participated  in  the  symposium  left  with  ideas  and  insight  for  improving  SA 
and  new  contacts  to  help  in  your  work.  I  hope  this  proceedings  document  proves  a  useful  reference 
for  SA  information  and  contacts. 


Meghan  Carmody-Bubb,  LT,  MSC,  USN 
Situational  Awareness  Integrated  Product  Team  Leader 
Code  4.6.4. 1,  Bldg.  2187,  Suite  2280 

Naval  Air  Warfare  Center  Aircraft  Division,  Patuxent  River,  MD  20670-1906 
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Haptics  As  The  Most  Intuitive  Spatial  Orientation  System 


Commander  Angus  H.  Rupert,  MC,  USN,  M.D.,  Ph.D. 
NASA  Johnson  Space  Center 
Naval  Aerospace  Medical  Research  L  aboratory 
5 1  Hovey  Road,  Pensacola,  FL  32508-1046 


Background 

Given  the  criticality  of  a  high  level  of  situation  awareness  (SA)  for  survival,  the  pressures  of  evolution  have 
prov  ided  every  successful  species  with  multiple  sensory  systems  to  assure  a  keen  sense  of  SA.  All  animals  use  a 
combination  of  exteroceptive  sensors  (e.g.  visual,  auditory,  olfactory  and  somatosensory  including  tactile)  to  keep 
track  of  objects  in  the  external  env  ironment  and  proprioceptive  sensors  (e.g.  muscle  and  tendon  stretch,  vestibular 
linear  and  angular  acceleration  and  touch  receptors)  to  provide  awareness  of  body  orientation  and  dynamics 
within  the  environment.  Touch  spans  both  categories  of  sensors  which  together  provide  during  our  day-to  day 
activities  independent,  complementary  ,  concordant,  redundant,  reliable,  and  veridical  sources  of  information  that 
are  assimilated  and  integrated  in  the  central  nervous  system  (CNS)  to  develop  spatial  orientation  and  SA. 
Obviously  spatial  orientation  is  a  necessary  prerequisite  for  SA.  Over  several  million  years,  man  and  other 
terrestrial  species  have  developed  a  refined  set  of  sensors  which  are  finely  matched  to  the  dynamics  of  our  normal 
daily  activities  in  the  two  dimensional  environment  for  which  they  were  “designed". 

The  sensory  systems  that  serve  so  well  on  the  ground  fail  when  exposed  to  the  acceleration  dynamics  of  the  high 
speed  platforms  in  the  aerospace  environment  The  frequent  changes  in  acceleration  and  direction  of  aircraft 
motion  subject  aircrew  to  a  resultant  gravitoinertial  force  that  is  constantly  changing  in  magnitude  and  direction. 
Under  such  circumstances,  the  somatosensory  and  vestibular  sensors  responding  to  this  constantly  varying 
apparent  gravitational  field  provide  concordant  but  false  information  concerning  the  direction  of  “down." 
Unfortunately,  varying  gravitoinertial  force  fields  can  also  produce  visual  illusions  of  motion  and  position.  Thus 
in  unusual  acceleration  environments  the  CNS  has  the  added  responsibility  of  determining  which  sensory 
information  is  valid 

Understandably,  the  typical  spatial  disorientation  mishap  occurs  when  the  visual  orientation  system  is 
compromised  (e.g.  temporary  distraction,  increased  workload,  transitions  between  visual  and  meteorological 
conditions,  or  reduced  visibility).  The  CNS  must  then  compute  orientation  with  the  only  information  remaining  - 
the  frequently  false  vestibular  and  somatosensory'  information,  which  is  however  in  agreement  with  each  other 
and  hence  very  compelling.  It  is  for  this  reason  that  under  these  circumstances  spatial  disorientation  is  a 
physiologically  normal  response 

TECHNOLOGY  Is  The  Situational  Awareness  Threat 

1)  Technology  has  prov  ided  platforms  that  expose  our  biological  sensors  to  stimuli  that  exceed  the  limits  of 
sensor  design. 

2)  Technology  now  permits  the  presentation  of  so  many  sources  and  such  large  quantities  of  information  that 

the  cognitive  integrative  capacity  of  the  CNS  is  overwhelmed. 

3)  The  dynamics  of  the  changing  information  and  conditions  provided  by  the  new  platform  technologies  far 
exceeds  the  processing  limitations  of  biological  systems. 

4)  Technology  has  increasingly  provided  aircrew  with  the  opportunities  to  fly  under  conditions  that  arc 
conducive  to  producing  spatial  disorientation  and  loss  of  SA. 

The  US  Army  has  noted  an  increase  in  the  number  of  spatial  disorientation  mishaps  since  1985  coinciding  with 
the  introduction  and  widespread  use  of  night  vision  goggles  (NVGs)  (I).  This  technology  permitted  new  mission 
profiles  including  the  introduction  of  night  nap-of-the-earth  flight,  night  formation  flight,  all  weather  flying  and 
the  carry  ing  of  external  loads  at  night  all  of  which  subject  the  pilot  to  more  opportunities  to  experience  spatial 


disorientation.  Furthermore  new  helicopters  are  more  agile  permitting  more  vigorous  acceleration  maneuvers  - 
another  factor  responsible  for  the  increasing  incidence  of  spatial  disorientation  mishaps. 

The  Solution  for  Aerospace  Spatial  Disorientation:  TECHNOLOGY 

The  solution  to  the  biological  problems  posed  above  is  technology. 

To  maintain  spatial  orientation  in  the  three  dimensional  aviation  environment,  pilots  must  be  provided  with  an 
intuitive,  continuous,  veridical  sourcc(s)  of  orientation  information  requiring  minimal  cognitive  effort  similar  to 
the  situation  humans  normally  experience  in  the  two  dimensional  terrestrial  environment. 

The  Tactile  Situation  Awareness  System  (TSAS)  is  simply  a  matrix  of  tactile  stimulators  (tactors)  incorporated 
into  a  flight  suit  (tactor  locator  system)  that  provides  the  pilot  with  critical  flight  parameters  via  the  sense  of  touch 
A  variety  of  algorithms  have  been  developed  that  apply  information  derived  from  the  aircraft  sensors  via  an 
intuitive  haptic  display  to  the  aircrew  so  that  they  are  continuously  provided  true  orientation  information.  The 
success  of  TSAS  as  an  orientation  device  is  described  in  other  papers  in  this  symposium  and  elsewhere 
(2,3, 4, 5, 6, 7)  What  has  made  TSAS  successful  is  the  intuitive  nature  of  the  display  . 

The  concept  of  using  skin  to  receive  information  normally  presented  by  the  visual  channel  is  not  new.  For 
example,  the  presentation  of  letters  via  Braille  is  used  by  thousands  of  visually  challenged  people.  The 
disadvantage  of  Braille  is  the  duration  of  the  training  period  required  to  become  proficient  in  its  use.  Like  reading 
for  children,  the  acquisition  of  Braille  skills  is  non-intuitive  and  typically  requires  at  least  a  year  or  two  of 
intensive  training.  In  the  late  1  Q60’s  and  early  I970's  there  were  several  efforts  (8,9)  to  provide  visual  information 
(pictures,  movies,  television  etc.,)  hapticly  on  the  back  or  chest  using  large  scale  dense  matrix  arrays  of  tactors.  In 
the  early  1 970’s  haptic  aviation  displays  for  navigation,  airspeed  control  (10),  or  command  direction  to  move 
controls  were  attempted  without  success.  None  of  these  attempts  succeeded,  in  part  due  to  the  non-intuitive  nature 
of  the  display  and  in  part,  due  to  presenting  information  that  exceeded  the  physiological  capabilities  of  the  touch 
sensory  system.  Skin  on  the  torso  has  a  two-point  discrimination  of  several  centimeters,  which  prevents  the 
presentation  of  information  that  does  not  match  the  bandw  idth  of  the  limited  resolution  available. 

Intuition  is  the  power  of  attaining  direct  knowledge  or  cognition  without  rational  thought  and  inference.  What 
makes  a  tool  or  system  intuitive  and  how  can  you  ascribe  a  value  to  the  “intuitiveness”  of  a  system?  An  interface 
or  system  is  intuitive  when  the  logical  operation  is  consistent  with  the  operator's  life  experience  and  mental 
model.  Most  pilot  interfaces  and  avionics  systems  arc  designed  by  engineers  and  human  factors  engineers  with 
the  view  of  training  the  pilot/operator  to  think  like  the  system  they  have  designed  The  recent  trend  in  human- 
centered  design  is  instead  to  develop  a  system  that  operates  like  the  pilot  thinks.  However,  despite  marginal 
improvements,  since  these  displays  are  visual  they  continue  to  require  significant  cognitive  effort.  TSAS  as  we 
shall  see  goes  one  step  further  in  that  it  works  like  the  pilot  reacts  by  accessing  the  pilot  at  the  level  of  “reaction'’ 
and  “subconsciousness”  behavior. 

Spatial  orientation  in  our  daily  activities  is  an  automated  process  that  does  not  normally  require  conscious  effort 
Nature  has  developed  highly  efficient  automated  “subroutines”  that  provide  excellent  spatial  orientation  to 
animals  while  permitting  them  to  attend  to  other  survival  tasks  such  as  hunting  or  evading  predators.  If  it  were 
necessary  for  prey  animals  to  devote  the  same  effort  to  orientation  that  pilots  currently  do  they  would  not  cat 
regularly  and  would  themselves  become  hunted. 

Reflexes,  such  as  limb  withdrawal  in  response  to  a  painful  stimulus,  occur  at  the  lowest  level  of  the  CNS 
organization  —  at  the  segmental  level  of  the  spinal  cord.  Such  reactions  have  limited,  well  defined,  constrained 
neural  elements  that  convey  robust  “hardwired”  responses  that  posses  limited  ability  to  change  with  repeated 
experiences.  By  comparison  high  level  cognitive  tasks  (e.g.,  instrument  reading  and  subsequent  decision  making) 
are  associated  with  cortical  processes  characterized  as  anatomicly  diffuse,  and  subject  to  behavioral  plasticity 
from  many  subtle  spheres  of  influence.  In  terms  of  computer  technology  reflexes  resemble  hardware  while 
cognition  resembles  software. 
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Between  these  two  extremes  is  the  organization  of  automated  functions  such  as  orientation.  The  lowest  level  at 
which  we  see  a  coordinated  orienting  reflex  in  response  to  local  stimulation  is  at  the  level  of  the  rrudbrain. 
Midbrain  stimulation  produces  a  patterned  eye,  head,  neck,  torso  and  limb  movement  to  direct  attention  to  a 
specific  location  in  space.  The  organization  and  temporal  development  of  the  neural  architecture  responsible  for 
these  orientation  reflexes  reflects  the  primacy  of  touch  for  spatial  orientation  and  situation  awareness.  In  the 
ontogeny  (i.e.,  developmental  sequences  in  the  life  of  the  individual)  of  neural  development,  the  first  sensory 
system  to  which  reactivity  can  be  shown  is  the  skin-muscle-joint  proprioceptive  system  (11).  As  can  be  seen  in 
Figure  1,  the  vestibular  system  follows  and  it  is  only  much  later  that  the  auditory  and  visual  orientation  sensory 
systems  arc  sequentially  added  to  form  the  overall  spatial  orientation  system  This  is  in  keeping  with  the  principle 
that  “ontogeny  recapitulates  phytogeny"  (i.c.,  individual  development  follows  the  temporal  pattern  that  has 
occurred  over  the  evolutionary  time  scale).  It  thus  indicates  that  touch  is  the  first  orientation  system  from  an 
evolutionary  perspective. 
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Figure  1  Timetable  outlining  sensory  development  of  the  domestic  cat.  (Turner  and  Bateson,  1 988) 

Within  the  midbrain  the  haptic  system  is  topologically  arranged  to  map  to  the  external  environment.  It  is  the 
lowest  layer  and  sequentially  the  auditory  and  visual  representations  of  the  world  are  overlaid  during  development 
with  their  connections  and  architecture  built  on  the  base  architecture  established  by  the  sense  of  touch.  For  this 
reason  touch  is  the  primal  orientation  system.  The  other  sensory  systems  depend  on  it  for  development.  Although 
people  are  often  bom  and  survive  without  vision  or  without  hearing,  individuals  bom  without  proprioceptive 
touch  do  not  survive.  Indeed  the  very  rare  individual  who  loses  the  sense  of  proprioceptive  touch  later  in  life  finds 
the  simple  task  of  reaching  for  objects  or  walking  so  cognitively  demanding  as  to  be  totally  exhausting  to  the 
point  that  remaining  in  a  wheelchair  may  be  the  better  choice.  This  should  cause  us  to  reevaluate  the  literature  that 
so  often  claims  vision  is  the  most  important  orientation  sense. 

Proprioceptive  touch  like  the  vestibular  system  provides  continuous  information  that  does  not  reach  our  level  of 
awareness  or  consciousness  except  under  unusual  circumstances  (e.g.,  conflicting  or  confusing  information).  In 
summary,  the  automated  function  of  haptic  orientation  has  a  well  defined  midbrain  architecture  (hardware),  fast 
robust  responses,  and  minimal  demands  on  cognition  which  make  it  the  best  candidate  to  provide  continuous, 
intuitive  orientation  information. 

One  measure  of  the  “intuitiveness"  of  a  tool  or  system  is  the  time  required  to  train  an  individual  to  use  the  system. 
A  perfectly  intuitive  system  requires  no  training.  For  example,  a  TSAS  designed  to  tap  the  operator  on  the 
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shoulder,  or  a  location  on  the  torso,  to  represent  a  threat  towards  which  shelie  is  to  direct  attention  is  a  system  that 
the  takes  advantage  of  the  cumulative  life  experience  of  the  operator.  In  actuality,  the  operator  has  been  “trained" 
from  the  point  in  development  at  which  these  reactions  could  be  demonstrated  in  the  womb.  Consistent 
reinforcement  of  this  reflex  with  day-to-day  life  experiences  provides  the  training  that  TSAS  takes  advantage  of 
through  human-centered  design  of  the  TSAS  interface.  Another  example  is  the  TSAS  orientation  to  the  direction 
“down"  -  a  prerequisite  for  both  visual  and  instrument  flight.  The  TSAS  provides  tactile  stimuli  to  the  area  of  the 
torso  where  the  pilot  would  normally  receive  pressure  cues  on  the  ground  if  she  tie  were  firmly  attached  to  a  chair 
with  multiple  straps.  Such  a  presentation  requires  only  minutes  to  train  either  an  experienced  pilot  or  novice  since 
for  both  individuals  the  logical  operation  is  consistent  with  their  mental  model  based  on  cumulative  life 
experience.  The  relation  between  intuitiveness  and  time  required  to  train  is  obvious.  Orientation  displays  that 
require  little  or  no  training  take  advantage  of  neural  architecture  developed  over  millions  of  years.  When  a 
designer  can  use  the  fast,  robust  reflexive  tactile  system  in  lieu  of  the  slower,  more  plastic  cognitive  visual 
processes  the  resulting  display  will  enhance  pilot  performance  by  increasing  the  cognitive  reserve  available  for 
other  cockpit  tasks. 

As  pointed  out  earlier,  some  aspects  of  SA  including  spatial  orientation  are  not  normally  cognitive  tasks 
demanding  attention.  They  occur  at  a  lower  level,  sometimes  reflexively,  and  operate  on  neural  architecture 
reflecting  the  development  of  “subroutines"  that  permit  the  individual  to  allocate  cognitive/attentional  resources 
to  other  tasks.  In  the  typical  military  cockpit  there  are  many  types  of  information  that  can  only,  or  are  best, 
presented  visually.  TSAS  presentation  of  flight  information  will  reduce  “clutter"  on  visual  displays  thereby 
enhancing  the  presentation  of  visual  data. 

The  TSAS  is  not  limited  to  individual  awareness  but  is  easily  applied  collectively  to  the  flight  crew.  Normally 
critical  information  is  either  duplicated  as  redundant  instrumentation  or  alternatively,  is  located  in  the  shared  space 
between  the  pilots  as  an  “open  interface"  whereby  pilots  monitor  the  action  of  the  other  without  verbal 
communication.  TSAS  can  provide  simultaneously  to  all  aircrew,  the  critical  information  (attitude,  airspeed, 
ground  proximity,  target  range  and  bearing,  etc.)  to  enhance  system  robustness  through  multiple  representation.  It 
will  also  enhance  detection  of  autopilot  errors  by  providing  all  aircrew  with  attitude  information  to  enable  aircrew 
to  detect  uncommanded  flight  profiles. 

Recommendations  for  Future  Research  Directions 

1)  Basic  science  research  focused  on  a)  optimizing  the  presentation  of  flow  and  dynamics  of  orientation,  b) 
techniques  to  minimize  the  number  of  tactors  using  the  illusions  present  in  the  system  of  touch,  and  c) 
mixtures  of  tactor  clusters  presenting  vibratory,  electrical,  and  tangential  sweep  stimuli. 

2)  Tactor  development  including  reduction  in  size  and  power  requirements. 

3)  Basic  science  research  to  determine  the  optimal  mixture  of  sensory  modalities  to  convey  spatial 
orientation  and  situation  awareness  information  in  the  aerospace  environment 
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Evaluation  of  Prototype  Display  of 
Enemy  Launch  Acceptability  Region  (LAR)  on  the  F/A-18  HUD 


LT  Meghan  A.  Carmody-Bubb  and  LCDR  Dave  A.  Maybury 
Naval  Air  Warfare  Center  Aircraft  Division 
Patuxent  River,  MD 

This  study  began  with  an  informal  survey  of  tactical  aircrew  for  suggestions  on  potential  display  modifications 
to  improve  situational  awareness  (SA)  in  the  electronic  air  combat  environment.  One  suggestion  that  was  repeated 
among  F/A-18  aircrew  was  to  display  an  enemy’s  Launch  Acceptability  Region  (LAR),  in  addition  to  ownship 
LAR,  which  is  currently  displayed.  The  current  Launch  Acceptability  Region  is  displayed  on  the  Normalized  In- 
Range  Display  (N1RD)  /  Allowable  Steering  Error  (ASE)  circle  on  the  head  up  display  (HUD),  as  well  as  on  the 
Radar  Scope.  It  provides  the  pilot  w ith  information  on  his/her  ability  to  destroy  a  threat.  The  LAR  includes  three 
parameters:  Rmax  (the  pilot  is  within  range  to  destroy  the  threat  with  the  selected  missile),  Rno  escape  (if  the  pilot 
fires  a  missile,  it  will  most  likely  destroy  the  threat),  and  Rmin  (if  the  pilot  shoots  after  Rmin,  the  missile  will  not 
arm  fast  enough  to  destroy  the  target).  At  the  present  time,  there  is  no  display  telling  the  pilot  when  he/she  is  within 
lethal  range  of  a  threat  aircraft;  e.g.,  when  he/she  is  in  the  threat’s  Rmin,  Rmax  and  Rno  escape.  The  pilot  must 
calculate  this  information,  considering  prior  information,  training,  intelligence  regarding  the  threat,  and  it's  probable 
missiles  and  their  parameters.  According  to  aircrew,  this  involves  a  good  deal  of  “number  crunching’’  headwork. 
While  aviators  are  skilled  at  such  tasks  and  often  have  the  ability  to  learn  weapon  performance  parameters  rapidly, 
it  takes  experience  and  flight  hours  in  a  particular  platform  to  develop  what  Schneider  and  Shiffrin  (1977)  describe 
as  automatic  processing  of  the  information.  According  to  Fisk  and  Scerbo  (1987),  there  are  several  fundamental 
differences  between  automatic  and  controlled  processing.  Whereas  controlled  processing  is  serial  in  nature, 
requires  effort,  and  requires  little  or  no  practice  to  achieve  asymptotic  performance,  automatic  processing  is  parallel 
in  nature,  requires  little  or  no  effort,  but  requires  extensive,  consistent  training  to  develop.  Perhaps  most 
importantly,  automatic  processing,  unlike  controlled  processing,  is  not  limited  by  short-term  memory  (STM) 
capacity.  Automatic  processing  of  the  information  inherent  in  determining  an  enemy’s  LAR  is  critical  to  a  pilot 
operating  under  the  stress  of  combat  conditions,  particularly  in  a  cockpit  that  is  becoming  increasingly  complex, 
with  multiple  sources  of  data  to  be  manipulated  in  STM.  Wrhile  automatic  processing  might  be  expected  in  the 
platform-seasoned  aviator,  many  combat  aviators,  such  as  a  pilot  fresh  out  of  the  Fleet  Replenishment  Squadron 
(FRS),  will  not  posses  that  level  of  familiarization  with  a  particular  platform 

The  purpose  of  the  present  study  was  to  examine  whether  the  concept  of  displaying  the  enemy’s  LAR  to  the 
F/A-18  pilot  would  significantly  enhance  his/her  SA  of  ownship  vulnerability  and  improve  performance  by 
reducing  number-crunching  headwork  and  allowing  for  more  timely  and  accurate  decisions  and  tactical 
maneuvering.  A  prototype  display  of  enemy  LAR  was  presented  on  a  simulated  F/A-18  HUD,  and  compared  with 
the  current  LAR  display.  Results  indicated  that  pilots  were  more  accurate  in  determining  their  own  vulnerability 
with  the  prototype  display,  and  that  they  did  so  more  rapidly.  Additionally,  subjective  data  indicated  that  pilots 
perceived  themselves  as  having  better  SA  and  lower  workload  with  the  prototype  display. 

Method 

Eleven  test  pilots  from  Strike  Aircraft  Test  Squadron  at  the  Naval  Air  Warfare  Center,  Aircraft  Division, 
Patuxent  River  participated  as  subjects.  All  had  time  piloting  the  F/A-18,  including  6  qualified  F/A-18  pilots.  The 
study  was  conducted  in  the  Crewstation  Technology  Laboratory  (CTL)  and  utilized  the  behavioral  and  testing 
resources  developed  and  operated  by  Dr.  R,  S.  Dunn  and  his  staff.  Special  contributions  were  made  by  Tom 
Moulds  of  the  CTL,  who  spent  many  hours  programming  and  reprogramming  the  various  prototype  displays  and 
experimental  scenarios. 

The  simulator  used  for  the  testing  was  a  mid-fidelity  F/A-18  mimic  (did  not  utilize  actual  mission  computer) 
developed  by  Sim  Systems  and  maintained  and  modified,  where  applicable  and  allowable,  in  the  CTL. 

Prior  to  data  collection,  all  subjects  received  a  preflight  brief.  The  brief  included  a  “familiarization  flight”  in 
the  simulator,  the  goals  of  the  mission  and  the  evaluation,  and  the  parameters  of  the  notional  missiles  carried  by 
ownship  and  opponent  aircraft,  described  below. 
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Each  subject  flew  5  scenarios  that  varied  in  altitude  and  airspeed  All  scenarios  began  40  miles  out  and  ran 
until  the  clock  on  the  ASE^NIRD  circle  reached  just  past  Rmax  All  scenarios  were  flown  first  w  ith  the  current 
LAR  display  (F’gure  I ) ,  and  then  with  the  pr  'otype  display  of  enemy  I  A  R  (Figure  2). 


Figure  1:  Current  LAR  Display 

(Clock  ticks  down  counterclockwise.  In  this  instance,  ownship  is  approaching  Rmax.) 


Figure  2:  Prototype  LAR  Display 

(Square  box  represents  opponent’s  clock.  In  this  case,  the  opponent  is  at  Rmax  before  ownship; 
i.e.,  the  opponent  has  “shoot  first”  advantage.) 

This  aspect  of  the  design  was  not  counterbalanced.  As  all  subjects  were  familiar  with  the  current  display,  it 
served  essentially  as  a  baseline  and  the  experimenters  did  not  expect  its  precedence  to  have  an  effect  on  data  or 
ratings  under  the  prototype  display  condition.  On  the  other  hand,  it  was  expected  that  prior  viewing  of  the 
prototype  display  might  effect  either  performance,  verbal  reports,  or  subjective  ratings  of  the  current  display. 

Scenarios  were  designed  to  provide  an  equal  number  of  engagements  in  which  either  the  ownship  or  the 
opponent  had  the  “shoot  first”  advantage.  In  all  cases,  the  subjects  were  instructed  to  maintain  altitude  and  airspeed 
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(except  the  Combat  Air  Patrol  scenario,  in  which  they  were  directed  to  increase  speed  upon  radar  contact).  Subjects 
were  told  to  assume  their  opponent  was  co-altitude  and  co-airspeed,  in  a  straight-on  1  v  1  engagement.  Aircraft 
maneuvers  had  to  be  discouraged  due  to  limitations  of  the  simulator,  but  aircrew  were  instructed  to  verbally  report 
any  evasive  or  aggressive  maneuvers  they  would  pursue  in  any  given  scenario.  Suojects  were  also  instructed  10 
verbally  report  whether  they  or  their  opponent  had  the  “shoot  first”  advantage,  both  at  initial  contact  and  at  Rmax. 

The  simulator  was  programmed  with  parameters  for  two  different  notional  missiles:  one  for  ownship  and  one 
for  opponent  aircraft.  The  concept  of  using  a  notional  missile  was  applied  in  order  to  create  a  potential  for  memory 
load  similar  to  what  a  novice  (in  platform)  pilot  might  experience  under  combat  conditions.  Because  we  could  not 
recreate  this  scenario  in  a  simulation  using  highly  experienced  test  pilots  who  have  intimate  knowledge  of  the  F/A- 
18  weapons  payload,  we  created  notional  missiles  for  both  ownship  and  opponent  Each  subject  studied  the 
performance  parameters  of  both  missiles  prior  to  participation  in  the  simulator  runs. 

During  the  simulator  runs,  video  and  audio  data  were  recorded.  The  video  recorded  the  information  from  the 
simulator  screen  (the  HUD  information),  while  audio  recorded  pilot  comments  and  verbal  data.  Following  each 
round  of  scenarios  (without  versus  with  the  new  prototype  display),  subjects  were  given  the  China  Lake  Situational 
Assessment  (CLSA).  This  is  a  subjective  situational  awareness  scale  under  development  by  Steven  Adams  at  the 
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Figure  3:  CLSA  Scale  Results 


Naval  Air  Warfare  Center  Weapons  Division  (NAWCWD)  China  Lake.  The  descriptors  on  the  scale  were  modified 
slightly  to  fit  the  present  task,  A  sample  of  the  scale  is  provided  in  Figure  3.  Additionally,  after  all  simulator  runs 
were  completed,  each  subject  was  given  the  Subjective  Workload  Dominance  (SWORD)  and  SA-SWORD  tests,  a 
final  questionnaire  and  a  debrief.  It  should  be  noted  that  the  SWORD  and  SA-SWORD  were  modified  slightly. 
There  were  9  response  slots  between  the  2  displays  being  compared,  as  opposed  to  the  typical  17  slots.  According 
to  Vidulich,  Ward,  and  Schueren  (1991),  the  number  of  slots  was  adapted  directly  from  Saaty's  (  1980)  Analytic 
Hierarchy  Process  (AHP)  scale,  in  which  1 7  slots  were  selected  to  comply  with  Miller's  (1956)  seven  plus  or  minus 
two  capacity  of  working  memory  (1  slot  for  equal  and  8  on  each  side).  This  was  maintained  in  SWORD  because 
the  results  of  Budescu,  Zwick,  and  Rapoport  (1986)  showed  no  reason  to  alter  it  In  other  words,  there's  no 
evidence  the  absolute  value  of  the  slot  number  is  a  critical  element 

Results 

Video  and  audio  data  were  post- processed  to  obtain  accuracy  and  ‘'decision  time"  data.  Accuracy  was 
determined  by  the  subjects’  verbal  reports  of  whether  they  or  their  opponent  had  the  “shoot  first"  advantage  in  a 
particular  scenario.  The  percentage  of  correct  reports  across  scenarios  was  recorded  for  both  the  initial  report  (initial 
radar  contact  /  start  of  wind-down  clock  on  ASE/NIRD  circle)  and  the  report  at  Rmax  Decision  time  data  was 
somewhat  less  explicit.  Because  verbal  data  were  used,  there  was  not  a  specific  subject- initiated  event  (like  a  button 
press)  from  which  to  time  how  long  it  took  a  subject  to  decide  who  had  the  advantage.  Instead,  the  investigators 
used  “length  of  report."  Subjects  either  gave  an  immediate  report,  such  as  “I  have  the  advantage,"  or  they  appeared 

to  think  aloud,  giving  reports  such  as  “Now  at  this  altitude  and  airspeed . I  should  have  the  advantage."  Length 

of  the  report  or  “decision  time”  was  defined  from  the  first  verbalization  to  the  point  at  which  a  distinction  between 
ownship  and  opponent  was  made. 

In  analyzing  the  performance  data,  the  scenarios  were  treated  as  separate  trials  across  which  the  data  were 
averaged.  A  t-test  for  dependent  samples  was  performed  on  the  current  LAR  display  versus  the  prototype  LAR 
display  for  both  the  accuracy  and  the  decision  time  data.  Results  indicated  significant  improvements  in  accuracy  (p 
<  .05)  both  at  initial  and  at  Rmax,  and  significant  decreases  in  decision  time  (p  <  .05)  with  the  prototype  LAR.  The 
respective  means  are  displayed  in  figures  4,  5  and  6  below. 


Old  Lar  New  Lar 


Figure  4:  Mean  %  Correct  at  Initial 
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Figure  5:  Mean  %  Correct  at  Rmax 


Old  Lar  New  Lar 


Figure  6:  Answer  Length/Decision  Time  (In  seconds) 

Regarding  the  subjective  data,  the  Wilcoxon  Matched  Pairs  test  was  performed  on  the  CLSA  data.  It  was 
significant  at  p  <  .01  For  purposes  of  comparison,  the  mean  CLSA  rating  for  the  current  LAR  display  was  2.68, 
while  that  of  the  prototype  LAR  display  was  1 .45  (note  that  on  the  CLSA  scale,  the  lower  the  rating,  the  higher  the 
perceived  SA). 

A  t-test  for  dependent  samples  was  performed  on  both  the  SWORD  and  SA-SWORD  data.  As  there  is  some 
controversy  over  whether  SWORD  data  is  truly  ratio  data,  a  Wilcoxon  Matched  Pairs  test  was  performed,  as  well 
Both  tests  were  significant  for  both  sets  of  data  at  p  <  .01.  The  workload  rating  means  for  SWORD  were  .744  and 
.256  for  the  current  and  the  prototype  display,  respectively.  Those  for  SA  SWORD  were  .268  and  .732  for  the 
current  and  the  prototype  display,  respectively. 
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Overall  results  for  the  questionnaire  are  reported  on  the  chart  below: 


1  Mow  much  do  you  think  the  prototype  display 
of  enemy  LAR  would  enhance  vour  SA  of 
\our  susceptibility  to  a  threat'’’’ 

2  How  much  do  you  think  the  concept  of 
displaung  enemy  LAR  would  enhance  vour 
SA  of  your  susceptibility  to  a  threat'’ 

3  How  much  do  you  think  the  concept  of 
displaying  enemy  LAR  would  enhance  your 
mission  performance’’ 

4  Mow  much  do  you  think  the  concept  of 
dtplaying  enemy  LAR  would  enhance  your 
survivability'’ 


In  addition  to  the  four  rating-based  questions,  pilots  were  asked  the  open-ended  question,  tkWhat  changes,  if  any, 
would  you  like  to  see  in  the  display?”  The  most  frequent  responses  to  this  question  included  the  following: 

1)  Answers  related  to  the  actual  display,  such  as  the  desire  for  smoother  transitioning  of  the  tick-mark  (less 
jumpiness),  and  the  addition  of  aural  cues  (5  responses), 

2)  Desire  to  place  the  display  heads-down  in  addition  to  the  HUD,  such  as  on  the  radar  or  SA  page.  Most  of 
these  respondents,  however,  stated  that  if  they  had  to  choose  I  position  to  display  enemy  LAR,  it  would  be  on  the  HUD 
(4  responses). 

3)  Display  additional  information,  such  as  weapons  type  and  multiple  bogies  (4  responses). 

Discussion 

Whenever  a  new  concept  of  displaying  information  in  the  cockpit  is  introduced,  it  is  often  met  with  some 
resistance.  This  is  with  good  reason.  The  modem  cockpit  is  inundated  with  data  from  a  variety  of  sources,  and  whether 
or  not  those  data  are  transformed  into  usable  information  by  the  pilot  depends  on  several  factors  inherent  in  human 
cognition,  display  design,  and  their  interaction.  This  is  particularly  true  of  the  F/A-l  8,  with  its  single  crewmember  and 
its  very  utilized  mission  computer.  In  the  case  of  displaying  enemy  LAR,  for  example,  where  it  may  seem  obvious  that 
such  a  concept  should  improve  SA  of  ownship  vulnerability,  many  would  argue  that  any  additions  to  the  HUD  would 
prove  distracting  to  the  pilot.  The  approach  of  this  study,  therefore,  was  to  prove  the  utility  of  the  concept  of  displaying 
enemy  LAR  through  rapid  and  relatively  inexpensive  prototyping  .  Such  prototyping  can  be  a  very  useful  and 
important  method  of  obtaining  initial  data  to  examine  whether  or  not  a  new  concept  is  worth  pursuing. 

According  to  the  findings,  the  concept  of  displaying  enemy  LAR  on  the  HUD  significantly  improves  the  F/A-l  8 
pilot's  situational  awareness  of  ownship  vulnerability.  This  is  evidenced  by  improved  accuracy  and  reduced  decision 
time  in  determining  ownship  versus  opponent  advantage,  as  well  as  subjective  ratings  of  SA.  Furthermore,  the  subjects 
indicated  a  strong  belief  that  such  information  displayed  on  the  HUD  would  enhance  their  mission  performance  and 
survivability. 

From  this  point,  it  is  now  necessary  to  pursue  more  detailed  areas  of  the  research  question,  particularly  related  to 
displaying  the  information  under  the  various  operational  conditions  that  are  likely  to  be  encountered.  Future  research 
needs  to  examine  several  display  considerations  in  a  high-fidelity  simulation  of  realistic  operational  combat  conditions. 
Display  considerations  should  include  those  highlighted  by  the  aircrew,  including  placement  of  the  enemy  LAR  display 
(HUD  versus  radar  screen  versus  SA  page  versus  all  three),  optimal  design  of  the  display  itself,  and  the  optimal  number 
of  bogies  that  can  be  tracked.  The  research  field  is  currently  exploding  with  advanced  display  technologies,  from  3-D 
audio  to  virtual  reality  displays,  and  such  avenues  should  be  considered  for  their  potential  to  improve  enemy  LAR 
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displays.  Finally,  there  are  several  considerations  regarding  implementation  of  an  enemy  LAR  display  in  the  actual 
aircraft,  and  such  practical  matters  as  mission  computer  capacity  must  be  taken  into  account  when  operating  future 
evaluations  under  higher  fidelity  simulation,  and  well  as  in  any  fleet  implementation  planning. 
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The  Naval  Research  Laboratory  (NRL)  has  been  involved  in  research  and  development  efforts  in  the  areas  of 
(I)  automated  routing  of  strike  aircraft1,  (2)  advanced  3D  displays  and  interaction  technologies,  and  (3)  collaboration 
tools  and  protocols.  This  research  has  been  conducted  to  advance  the  state  of  the  art  in  each  technology  area, 
particularly  for  military  command  and  control  applications. 

This  paper  will  concentrate  on  the  research  conducted  in  item  (2),  the  results  of  which  are  incorporated  in  the 
STrike  Optimized  Mission  Planning  Module  (STOMPM).  The  STOMPM  testbed  allows  the  user  to  load  various 
terrain  data,  create  strike  related  routing  scenarios  consisting  of  threats  and  targets,  test  the  various  routing  algorithms 
and  assess  their  performance,  and  visualize  this  information  and  interact  with  it  in  a  natural  setting.  Two  versions  of 
STOMPM  currently  exist,  versions  1.0  (vl  0)  and  2.0  (v2.0).  The  STOMPM  vl.O  was  primarily  developed  to  test 
auto-routing  technology,  and  does  not  include  technology  needed  to  run  on  advanced  displays  such  as  the  Virtual 
Reality  Responsive  WorkBench,  VRRWB  (i.e.,  STOMPM  vl.O  operates  best  on  a  computer  monitor  with  a 
mouse/menu  interface).  The  STOMPM  v2.0  also  includes  autorouting  technology  (specifically  autorouting  algorithms 
which  take  into  consideration  fuel  and  turn  constraints  which  are  not  included  in  STOMPM  vl.O),  but  also  includes  a 
user  interface  that  is  well  suited  to  running  on  more  advanced  displays  such  as  the  workbench. 

Phis  paper  will  provide  a  high  level  description  of  both  versions  of  the  STOMPM  testbed.  Follow  ing  this 
discussion,  we  will  describe  state-of-the-art  technologies  in  visualization,  advanced  displays  and  scene  interaction 
capabilities  that  have  been  incorporated  within  STOMPM  v2.0.  We  will  conclude  with  a  discussion  of  the  advantages 
associated  with  the  use  of  the  workbench  and  also  the  interface  that  has  been  developed  within  STOMPM  for  the 
workbench.  Lastly,  we  will  discuss  areas  for  future  research. 

2.  STOMPM  SYSTEM  DESCRIPTION 

The  STOMPM  system  serves  as  a  testbed  for  the  research  and  development  of  strike  asset  routing  algorithms, 
3D  displays  and  interaction  techniques,  and  research  in  collaborative  tools  and  protocols.  Version  1.0  of  STOMPM 
was  built  mainly  to  support  research  conducted  in  strike  asset  routing  algorithms,  while  version  2  0  of  STOMPM  was 
built  primarily  to  support  advanced  3D  displays  and  interaction  techniques  (both  versions  have  similar  models,  e.g.. 
Radar  Terrain  Masking  (RTM)2.  The  following  sections  provide  details  associated  with  each  version  of  STOMPM. 

2.1  THE  STOMPM  vl.O 

The  original  STOMPM  testbed  (VI  .0)1  was  developed  in  C  using  the  FORMS4  software  library  for  the  user 
interface  and  the  native  SGI  graphics  library  for  rendering  the  scene.  The  main  emphasis  of  STOMPM  V 1 .0  is  to  allow 
the  developer  to  easily  incorporate  auto-routing  technology  and  be  able  to  test  the  algorithms  via  a  simple  mouse/menu 
user  interface.  Many  autorouting  algorithms  were  implemented  ranging  from  simple  least  cost  path  to  jointly  optimal 
routing1.  A  screenshot  of  the  STOMPM  system  is  shown  in  Figure  I  This  version  of  STOMPM  allows  the  user  to 
load  terrain  maps  (i.e.,  Digital  Terrain  Elevation  Data  or  DTED),  place  assets,  radar  types,  and  targets  on  the  terrain, 
specify  routing  parameters,  and  eventually  choose  a  particular  routing  routine  to  find  route(s)  from  the  assets  to  the 
targets.  Fhe  user  has  the  ability  to  save/load  scene  files,  view  the  environment  from  various  locations,  get/change 
information  about  entities  in  the  scene,  and  modify  certain  attributes  associated  with  the  visualization  of  this 
information. 
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Figure  1 :  Screenshot  of  STOMPM  V 1 .0  User  interface.  Shown  are  the  RTM  cones  and  a  set  of  suppression  and  attack 
routes.  The  suppression  routes  open  a  corridor  for  the  attack  routes. 

Although  version  1.0  of  STOMPM  was  specifically  designed  to  test  autorouting  technology,  near  the  latter 
part  of  it's  development  cycle  stereographies5,6  were  incorporated  into  the  interface.  Since  this  version  of  STOMPM 
was  still  based  on  the  mouse/menu  sty  le  interface,  it  did  not  work  well  with  stereographies  because  the  menus  were  not 
drawn  as  a  part  of  the  environment  being  modeled,  and  only  the  environment  and  the  objects  contained  in  it  were  in 
stereo.  This  was  part  of  the  reason  that  STOMPM  was  eventually  reimplemented.  A  second  reason  for 
reimplementation  was  due  to  the  fact  that  version  1.0  was  not  easily  extendable.  Creating  new  objects  meant  defining 
new  data  structures  for  objects.  An  object  oriented  paradigm  was  investigated  and  eventually  accepted  as  part  of  the 
design  in  order  to  provide  a  more  flexible  system  in  which  new  objects  could  be  created  with  little  effort,  making  use  of 
already  existing  object  classes. 

2.2  THE  STOMPM  v2,0 

The  STOMPM  V2.0  is  a  set  of  object-oriented  C++  toolkits  that  facilitate  the  development  of  virtual 
environment  simulation  &  planning  environments.  The  STOMPM  toolkits  as  they  exist  now  provide  a  means  to  access 
and  use  routing  algorithms,  visualize  and  interact  with  a  scene,  and  collaborate  with  other  distributed  STOMPM 
modules.  A  primary  goal  of  STOMPM  w  as  that  it  be  useful  and  easily  transferable  to  other  uses  or  designs.  Towards 
this  end,  each  of  the  STOMPM  toolkits  is  specific  in  purpose,  either  extending  existing  functionality'  or  adding  to  it. 
Through  the  combined  use  of  some  or  all  of  the  toolkits,  it  is  possible  to  easily  build  new  applications  and/or  interfaces 
by  adding  to  the  existing  foundation  of  components  and  coding  practices.  In  the  discussion  that  follows,  all  italicized 
words  can  be  interpreted  as  base  classes  or  objects  derived  from  those  base  classes  In  either  case,  the  meaning  of  these 
italicized  words  are  the  same  in  the  context  in  which  they  are  used,  the  difference  is  important  only  in  the  design  and 
implementation  phase  (i.e.,  they  can  be  used  interchangeably  for  the  purpose  of  the  discussion). 

The  purpose  of  STOMPM  is  to  support  the  capability  to  generate  an  automated  set  of  routes  from  a  set  of 
starting  points  (sources)  to  a  set  of  targets  (sinks)  contained  in  a  defined  scene  with  obstacles.  A  primary  goal  of 
STOMPM  V2.0  is  that  it  continues  to  be  an  extensible  application,  ready  for  use  or  as  the  basis  of  new  or  old 
applications.  In  achieving  these  two  goals,  as  with  all  programming,  it  is  important  to  define  the  constraints,  key 
dependencies,  and  post-conditions  that  define  how  to  implement  the  goals  more  concretely.  Therefore,  the  design  of 
STOMPM  started  with  the  desired  capabilities,  which  were  mainly  set  at  the  beginning,  but  also  evolved  over  the 
course  of  the  project.  Below  we  discuss  the  capabilities  developed  as  a  result  of  our  goals. 

The  routing  algorithms  in  STOMPM,  given  a  correctly  specified  scene  and  set  of  routing  constraints,  calculate 
an  optimal  route  from  a  source  to  a  target.  Currently  there  are  three  algorithms  in  V2.0,  but  this  number  will  be 
extended  in  the  future.  There  is  an  unconstrained  router,  a  router  that  is  restrained  by  tum-angles  but  trades  speed  for  a 
possibly  non-optimal  route,  and  an  optimal  turn-angle  constrainc  i  router.  The  algorithms  are  interested  in  those  objects 
in  the  scene  that  represent  a  threat  to  an  asset  attempting  to  get  from  the  source  to  the  sink  An  example  of  such  a  threat 
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is  a  radar,  which  interacts  in  complex  ways  with  the  surrounding  terrain.  Thus  the  router  is  dependent  upon  a  certain 
scenario  or  scene  the  user  has  created,  and  the  underlying  objects  that  make  up  this  scene. 

STOMPM  provides  several  different  utilities  that  may  be  used  to  create  a  versatile,  alterable,  and  extendable 
user  interface  to  interact  with  the  objects  in  a  scene  or  other  components  of  STOMPM.  The  first  point  regarding  the 
STOMPM  interface  is  it's  ability  to  provide  a  representation  of  the  scene  which  the  route-  will  use.  implying  both  a  u  t 
viewpoint,  and  graphic  representations  for  all  objects  that  make  up  the  scene  Visualization  is  crucial  for  concepts  like 
RTM  that  are  most  intuitively  understood  and  correctable  when  visualized.  Secondly,  the  ability  to  alter  the  user 
viewpoint  and  the  objects  in  the  scene  (hereafter  referred  to  as  SceneObjects)  is  present  in  the  form  of  a  user  HotSpot. 
The  HotSpot  is  a  3D  version  of  a  mouse  pointer  and  can  be  driven  by  a  variety  of  input  devices  (mouse,  6  degree  of 
freedom  tracker,  keyboard,  etc.)  and  is  used  to  select,  move  objects,  alter  the  user’s  view,  etc.  STOMPM  also  is  able  to 
store  and  retrieve  scenes  for  use  at  a  later  time. 

Though  the  interactions  with  the  HotSpot  can  be  varied  in  several  ways,  there  is  still  a  need  for  other  forms  of 
input  STOMPM  also  provides  both  keyboard  support  and  3D  menus  (which  interact  with  the  HotSpot).  All  of  this 
assumes  that  the  user  wishes  to  interact  directly  w  ith  the  scene.  However,  another  means  of  changing  the  scene  the 
router  uses  is  also  available  within  STOMPM  There  exists  support  for  distributed  communication,  currently  limiting 
the  users  to  one  concurrent  shared  scene.  Thus,  it  is  possible  to  alter  the  layout  of  the  scene  by  reading  from  a 
STOMPM  feed  coming  from  another  computer  over  a  network.  The  interactions  with  the  scene  are  merely  support  to 
supply  the  router  with  the  necessary  information  to  do  its  work. 

The  STOMPM  system  is  composed  of  a  complex  set  of  components.  It  is  able  to  prov  ide  scene  management, 
viewing,  support  for  various  input  devices,  a  3D  menuing  system  (Figure  2),  peer-to-peer  networking  support,  object 
interaction  through  the  HotSpot ,  provide  information  feedback,  archivability,  as  well  as  providing  hooks  for  other 
useful  operations.  The  next  section  will  describe  the  visualization,  advanced  display,  and  scene  interaction  capabilities 
that  have  been  implemented  within  STOMPM  v2.0. 


Figure  2:  The  STOMPM  v2.0  user  interface  showing  the  HotSpoX  (seen  as  circle  in  'Low"  per  object  submenu),  3D 
System  Menu,  and  3D  Per/Object  menus  for  manipulating  object  specific  information 


3.  VISUALIZATION,  ADVANCED  DISPLAYS  AND  INTERACTION  rN  STOMPM 

The  next  section  will  discuss  a  visualization  technology  that  has  been  investigated  and  implemented  within 
STOMPM,  namely  the  use  of  stereographies  for  the  inspection  of  the  3D  strike  information  The  following  section 
will  discuss  an  advanced  display  technology  being  utilized  for  the  3D  visualization  of  STOMPM  scenarios,  namely  the 
virtual  reality  responsive  workbench.  The  advantages  of  using  the  workbench  will  be  presented.  Lastly,  we  will 
discuss  novel  interaction  technologies  that  have  been  developed  within  STOMPM  for  the  workbench. 

3.1  VISUALIZATION  TECHNIQUES 

Stereographies5,6  provides  a  true  3D  representation  of  an  environment  by  providing  two  images  to  the  eyes  in 
sequential  order,  one  for  the  left  and  one  for  the  right.  This  allows  the  user  to  perceive  depth  on  a  two-dimensional 
monitor.  Implementing  the  stereo  effect  in  software  is  not  very  difficult  and  works  as  follows:  produce  two  images  of 
the  scene  and  double  the  monitor's  refresh  rate.  One  of  these  images  is  for  the  right  eye  while  the  other  is  for  the  left 
eye.  The  user  can  then  wear  Liquid  Crystal  Display  (LCD)  shutter  glasses  to  view  the  image  in  stereo.  The  shutter 
glasses  work  by  show  ing  the  left  eye  the  image  intended  for  the  left  eye  and  the  right  eye  the  image  intended  for  the 
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right  eye,  in  alternating  sequence  (i.e.,  the  shutters  in  the  glasses  open  and  close  in  synchronization  with  the  monitors 
refresh  rate  -  the  synchronization  signal  is  sent  to  the  glasses  from  an  emitter).  The  overall  effect  to  the  user  is  a  view 
which  more  closely  resembles  3D  -  the  stereo  image  can  be  either  projected  in  front  of,  or  behind,  the  computer  screen, 
by  adjusting  a  parameter  in  the  software  that  controls  the  distance  from  the  eyes  to  the  image  convergence  point. 

There  are  several '  *ms  worth  mentioning  about  the  use  of  stereo.  As  was  alread}  mentioned,  the  stereo  image 
can  be  projected  in  front  of,  or  behind,  the  computer  screen  by  adjusting  a  certain  parameter  in  the  software  When  one 
sets  the  parameter  such  that  the  image  is  projected  in  front  of  the  screen,  the  eyes  can  get  confused  by  a  floating  image 
in  front  of  the  screen,  which  when  seen  in  comparison  to  the  edges  of  the  window  display,  appear  underneath  the 
window/display  (edge  effects).  Zooming  or  panning  effects  can  further  magnify  the  “edge  effect”  phenomenon 
Therefore,  it  is  important  to  have  the  entire  scene  visible  when  one  w  ishes  to  project  the  image  in  front  of  the  computer 
screen.  However,  by  having  the  entire  scene  visible  on  the  screen,  it  may  be  impossible  to  view  the  important  details 
associated  with  the  scenario.  In  many  instances,  it  is  desirable  to  project  the  image  behind  the  screen.  What  we  have 
noticed  is  that  in  our  particular  application,  when  the  maximum  height  of  the  viewable  terrain  is  projected  behind  the 
screen,  even  after  panning  or  zooming,  edge  effects  arc  removed.  In  this  regard,  it  may  be  easier  to  zoom  or  pan,  thus 
enabling  more  details  associated  with  the  scenario  to  be  seen 

3.2  ADVANCED  DISPLAYS 


The  NRL  has  investigated  the  use  of  advanced  displays  for  Command  and  Control  applications,  particularly 
the  use  of  a  virtual  reality  responsive  workbench,  (Figures  3a  and  3b),  The  workbench  was  originally  developed  and 
built  at  the  GMD  National  Research  Center  for  Information  Technology,  and  a  copy  was  built  at  NRL  for  initial 
research.  Currently,  NRL  is  using  a  commercially  available  workbench  developed  by  Fakespace  Corporation. 
Whereas  the  original  workbench  top  was  not  adjustable,  the  one  developed  by  Fakespace  has  an  adjustable  table  top 
which  can  tilt  to  approximately  45  degrees  for  easier  viewing. 


Figure  3a:  A  schematic  of  the  virtual  reality  responsive 
workbench  (printed  from  the  NRL’s  VR  Laboratory 
homepage). 


Figure  3b:  A  computer  generated  image  of  the  virtual 
reality  responsive  workbench 


Video  from  the  computer  is  sent  to  the  projector,  which  projects  the  stereo  image  onto  a  mirror.  The  mirror 
reflects  this  image  onto  a  translucent  table  top  Two  pairs  of  emitters  are  mounted  at  the  back  Uvo  comers  of  the  table 
top.  The  use  of  two  pairs  of  emitters  provides  a  stronger  synchronization  signal  for  the  shutter  glasses,  however,  a 
single  emitter  could  be  configured  for  use.  When  displaying  an  image  on  the  workbench,  the  user  can  control  whether 
the  image  appears  to  float  above,  or  just  below,  the  table  top  by  adjusting  the  same  parameter  which  controls  where  the 
image  converges  (e.g.,  with  respect  to  the  far/near  clipping  planes). 

The  advantage  of  the  use  of  stereo  on  the  virtual  workbench  arises  from  observing  that  users  naturally 
perceive  altitude  in  the  same  direction  as  a  vector  which  is  perpendicular  to  the  earth.  The  workbench  provides  an 
environment  in  which  users  can  naturally  interact  with  objects  on  terrain  in  a  natural  table  top  environment.  On  a 
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computer  monitor,  altitude  would  be  in  the  same  direction  as  a  vector  perpendicular  to  the  computer  screen.  This  is  a 
little  awkward  to  work  with,  especially  when  trying  to  adjust  the  routes  in  the  z  direction.  The  viewing  area  is  also 
greater  as  compared  to  the  standard  computer  monitor.  This  larger  viewing  area  provides  a  more  comfortable 
environment  for  multiple  user  to  interact  with  the  workbench  application. 

3.3  INTERACTION  IN  STOMPMv2,0 

STOMPM  is  made  up  of  several  components  and  toolkits.  Each  toolkit/component  provides  an  independent 
functionality  that  is  intrinsically  different  from  the  others  The  primary  toolkits  that  STOMPM  provides  are  the 
Application  Tool  Kit  (ATK),  the  Routing  Tool  Kit  (RTK),  the  Graphic  ATK  (GATK),  and  the  Networking  TK.  Within 
each  of  these  toolkits  are  components  and  or  extensions  to  objects  in  other  toolkits.  Another  crucial  component  is  the 
SceneObject  that  the  other  toolkits  either  manage  or  use  to  accomplish  their  task. 

The  ATK  provides  a  framework  basis  for  the  other  toolkits.  The  GATK  adds  graphic  capabilities  to  the 
existing  system.  The  RTK  uses  the  ATK  and  SceneObjects  to  perform  its  routing  tasks.  The  Networking  TK  extends 
network  functionality  to  the  ATK  &  GATK,  but  could  also  be  considered  more  an  extension  of  the  existing  framework 
than  a  part  of  the  framework.  The  GATK  will  now  be  described  in  greater  detail. 

The  purpose  of  the  GATK  is  to  provide  a  base  set  of  tools  that  are  extendable  for  developing  the  GUI  that  will 
meet  the  end-users'  needs.  The  GATK,  as  its  name  implies,  inherits  from  the  ATK,  mainly  because  all  of  the 
components  it  uses  must  also  be  changed  to  be  graphic  in  nature  but  are  the  same  in  functionality.  A  platform 
limitation  of  the  current  implementation  is  that  the  GATK  uses  Performer,  an  SGI  rendering  system,  to  do  its  rendering 
and  much  of  its  interfacing.  Beyond  these  factors,  the  GATK  extends  the  ATK  to  include  input  devices,  viewing 
paradigms,  interface  mechanisms,  and  other  IO  components  which  are  usable  in  part,  whole,  or  not  at  all,  based  on  user 
requirements. 

The  key  extension  of  the  GATK  is  the  introduction  of  the  HotSpot.  The  HotSpot  corresponds  to  a  3D  mouse 
pointer  in  concept.  With  the  HotSpot ,  you  can  select,  pop  menus  up,  move  objects,  etc.  It  is  the  graphical  means  by 
which  input  is  specified.  Visually  it  is  represented  as  a  spot  on  the  screen,  which  is  simple  enough  to  understand.  The 
more  difficult  part,  is  how  to  move  and  position  it  with  6  degrees  of  freedom,  and  how  it  communicates  with  the  objects 
it  interacts  with. 

The  HotSpot  currently  is  driven  by  one  of  two  input  devices.  The  first  input  device  is  a  3-Button  mouse,  and 
the  second  input  device  is  a  6  degree  of  freedom  tracker.  The  tracker  is  currently  used  as  a  virtual  pointing  stick  to 
obtain  the  point  of  interest  on  the  screen  The  point  of  interest  for  a  mouse  is  likewise  an  x-y  coordinate  pair.  The  point 
of  interest  is  used  to  project  a  ray  from  the  users  viewpoint  to  the  viewing  screen's  position  in  world-space,  and  the  first 
object  of  intersection  (assuming  intersection)  becomes  the  location  of  the  HotSpot.  So  in  a  very  real  sense,  the  HotSpot 
is  its  own  device  that  is  driven  by  the  tracker  or  the  mouse.  Using  the  buttons,  the  tracker  or  mouse  handler  can  move 
the  HotSpot  in  or  out.  As  the  HotSpot  is  considered  native  to  the  GATK,  whereas  the  tracker/mouse  devices  are  not,  the 
HotSpot  has  communication  protocols  established  with  the  objects  it  interacts  with. 

Every  object  of  significance  on  the  display  is  given  the  ability  to  handle  events.  By  this  mechanism,  all 
GSceneObjccts  (and  InterfaceObjects  such  as  menus)  are  given  the  ability  to  respond  to  events  that  relate  to  them  This 
is  set  up  primarily  for  interactions  with  the  HotSpot ,  though  the  developer  has  the  option  of  extending  this.  The 
HotSpot  thus  at  agreed  times  sends  informational  messages  to  the  objects  it  interacts  with.  In  particular,  when  the 
object  is  selected,  unselected,  hit,  or  unhit,  the  object  in  question  is  notified  that  the  event  took  place.  Thus,  each  object 
chooses  how  it  will  respond  to  particular  events.  Most  objects  will  probably  respond  in  the  same  way,  and  thus  are 
given  a  default  handler.  For  instance,  when  most  SceneObjects  receive  a  move  event,  they  move  themselves  in  space. 
However,  when  the  Terr  a  inObject  receives  a  move  event,  its  behavior  is  overridden  to  move  the  viewer  of  the  scene 
thereby  accomplishing  the  desired  interaction  Thus  every  object  has  the  chance  to  easily  override  behavior. 

By  using  the  HotSpot  and  its  communication  protocol,  several  features  of  interest  have  been  added  to 
STOMPM  The  user  can  now  add  3D  menus  that  when  selected  will  perform  developer  specified  callbacks.  These 
menus  are  also  operable  on  a  per-object  basis,  radars  can  have  one  type  of  menu,  each  tank  can  have  its  own  specialized 
menu,  etc.  Since  these  menus  are  just  like  any  other  3D  object  in  the  scene,  the  stereo  effect  is  preserved.  Objects  can 
have  designated  common  handlers,  e.g.  all  objects  that  must  be  placed  on  the  ground  can  use  one  common  handler  that 
drops  the  object  back  to  the  ground  when  left  in  the  air,  while  objects  that  can  be  left  in  the  air  use  a  different  handler 
An  important  aspect  of  a  good  user  interface  is  the  ability  to  move  through  the  displayed  scene  quickly  and  easily. 

STOMPM  currently  has  two  chief  modes  of  viewing  the  scene.  These  modes  are  tethered  viewing  and 
egocentric  viewing,  which  are  entered  into  by  selecting  the  menu  items  under  the  viewing  menu.  In  tethered 
viewing  mode,  the  user  is  virtually  tethered  to  the  3D  point  of  interest  in  world  space,  which  is  where  the  HotSpot 
was  before  entering  viewing  mode.  In  this  case,  moving  to  the  left  entails  rotating  about  the  3D  point  of  interest  at  a 
fixed  radius  in  the  XY  plane  that  corresponds  to  the  viewer’s  left.  Viewing  in  egocentric  mode  is  simpler.  When 
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the  camera  “moves'’  to  the  left,  the  viewer  rotates  their  view'  and  does  not  translate  at  all  Tethered  viewing  mode  is 
of  primary  use  when  examining  an  object  from  several  angles,  but  always  looking  towards  the  same  area. 
Egocentric  viewing  mode  is  always  looking  from  the  same  area.  A  third  useful  application  of  egocentric  viewing  is 
the  ability  to  jump  to  any  object  the  HotSpot  points  to,  and  then  view  from  that  point.  Thus,  it  is  possible  to  jump  to 
a  pilot's  view  or  a  view  from  a  certa:  hilltop. 

By  using  these  two  interface  mechanisms,  and  other  aspects  of  the  GATK  and  GSceneObject  interface  it  is 
possible  to  quickly  maneuver  through  a  scene  and  view  its  objects,  as  well  as  relocate  them.  Currently  missing  from  the 
framework  is  an  informational  feedback  of  where  the  objects  are  as  they  are  being  moved,  though  this  could  be 
implemented  quickly  enough.  Other  features,  such  as  drop-lines  from  the  objects  to  give  the  user  positioning 
information  when  they  float  above  the  terrain,  bins  to  temporarily  place  objects  in,  means  of  removing  all  objects  of  a 
given  type,  etc.  are  all  possibilities  with  the  current  STOMPM.  However,  as  may  have  been  apparent  from  the 
discussion  there  are  some  features  that  are  more  a  part  of  the  toolkit  than  others,  and  it  is  questionable  w  here  to  draw  the 
line  between  the  toolkit  components  and  objects  proper,  and  those  specific  to  the  application.  The  end  choice  is  always 
up  to  the  developer. 

4.  CONCLUSION 

The  visualization  of  strike  related  information  such  as  aircraft  routes,  terrain  and  radar  envelopes  has  been 
enhanced  by  the  use  of  stereographies,  which  has  made  it  easier  to  view  depth.  Furthermore,  coupling  stereographies 
with  an  advanced  display  technology  such  as  the  workbench  allow  the  users  to  work  with  a  true  3D  representation  of 
the  world  on  a  table  top  environment,  which  appears  to  be  most  natural  for  planners  as  many  of  the  planning  activities 
with  maps,  etc  are  done  on  table  tops.  The  workbench  also  provides  a  greater  viewing  area,  allowing  multiple 
participants  to  view  and  potentially  interact  with  the  environment  being  modeled.  The  interface  that  we’ve  developed 
within  STOMPM  works  well  with  the  virtual  workbench  in  many  respects.  Because  the  menu  system  is  3D,  it  does  not 
interfere  with  the  stereoscopic  workbench  application  as  would  the  traditional  menus.  Secondly,  the  object  reachability 
constraints  make  it  more  convenient  to  drive  the  HotSpot  via  a  joystick  as  opposed  to  driving  it  via  other  devices  on  the 
workbench  such  as  data  gloves.  Due  to  the  large  display  area  associated  with  the  workbench,  a  person  wearing  a  data 
glove  may  find  it  difficult  to  interact  with  objects  placed  over  such  a  wide  display  area.  Using  the  joystick  and 
associated  buttons,  the  user  is  able  to  rapidly  point  and  click  with  the  HotSpot  anywhere  on  the  display  area. 

5.  FUTURE  DIRECTION 

The  GATK  has  a  wealth  of  opportunities  for  expansion.  It  currently  stores  none  of  its  parameters  to  file,  so  it 
is  not  customizable  at  all.  There  are  many  concerns  regarding  the  display  of  other  information,  and  options  that  are 
possible  in  the  VR  world  that  go  beyond  the  components  common  to  the  X  environment  and  personal  computer 
operating  environments.  It  would  also  be  interesting  to  provide  X  widgets  and  menus,  and/or  porting  to  non-SGI 
systems,  which  would  require  a  hefty  rewrite  of  all  the  graphical  elements.  A  potential  area  for  investigation  is  the  use 
of  web  technologies  such  as  Java/  Virtual  Reality  Markup  Language  (VRML)  as  a  graphical  front  end  for  STOMPM. 
This  would  allow  planners  to  interactively  plan  and  collaborate  with  workbench  planners.  We  have  replicated  a 
module  within  STOMPM  using  Java  and  plan  to  continue  research  in  the  use  of  this  emerging  technology  for  real 
applications.  Another  area  that  we’re  investigating  is  the  potential  for  the  use  of  immersive  environments  in 
conjunction  with  the  workbench.  A  hypothesis  is  that  planning  functions  would  work  best  in  an  environment  which 
allows  “Gods  Eye”  viewing  such  as  that  provided  by  the  workbench.  Also,  once  generated,  these  plans  would  best  be 
simulated  in  immersive  environments.  We  are  actively  looking  into  issues  pertaining  to  the  interface  between  these 
two  different  display  platforms  We  are  also  investigating  multi-modal  interfaces  to  replace  or  complement  the  already 
existing  interface  within  STOMPM.  A  multimodal  interface  will  be  of  particular  importance  in  immersive 
environments  in  which  there  may  be  a  varied  and  large  amount  of  data  to  navigate  through. 
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Pilot-Vehicle  Interface  Adaptation 
Based  on  Situation  and  Workload 


Gerard  Rinkus,  Sandeep  Mulgund,  Karen  Harper,  and  Greg  Zacharias 
Charles  River  Analytics 
Cambridge,  MA 

We  describe  a  pilot/vehicle  interface  (PV1)  adaptation  system  that  uses  a  computational  situation  assessment 
(SA)  model  and  pilot  workload  metrics  to  drive  the  content,  format,  and  modality  of  military  cockpit  displays,  as  well 
as  modulate  the  degree  of  automaticity  of  various  vehicle  functions.  Our  conceptual  design  integrates  two  key 
information  streams:  1)  a  “content"  path,  driven  by  a  tactical  situation  assessment  module  that  uses  avionics  system 
outputs  to  determine  the  aircraft’s  current  condition  and  the  pilot's  information  needs  based  on  the  assessed  situation; 
and  2)  a  “format”  path,  which  uses  an  estimate  of  the  pilot's  state  (workload  level,  attentional  focus,  etc.)  to  determine 
the  most  appropriate  form  at/ modal  it>  for  presenting  the  needed  information  to  the  pilot.  Both  streams  are  also  used  to 
control  the  degree  of  automation  of  vehicle  functions,  e.g.,  defensive  countermeasures,  communication  tasks,  etc.  The 
resulting  PV1  maximizes  the  pilot's  situation  awareness  by  filtering  out  situationally  irrelevant  information,  makes 
optimal  use  of  the  pilot's  parallel  channels  of  information  processing  by  using  his  multiple  sensory  modalities  to 
transfer  information  between  pilot  and  vehicle,  and  reduces  the  pilot's  workload  by  automating  less  critical  tasks  during 
dangerous  and  demanding  periods. 

file  system  is  designed  modularly  to  facilitate  testing  of  various  computational  formalisms  for  the  individual 
components.  The  system  currently  uses  Bayesian  belief  networks  for  the  situation  assessment  and  mental  workload 
estimation  subsystems  and  a  rule-based  system  for  the  PV1  adaptation  subsystem.  The  mental  workload  model 
incorporates  several  physiological  variables,  including  heart  rate  and  heart  rate-derived  measures,  EEG-derived 
measures  and  eyeblink  measures.  The  belief  network's  quantitative  relations  were  derived  using  results  from  correlation 
studies  in  the  mental  workload  literature.  While  narrow  in  scope,  the  network  produces  intuitively  sensible  results  for  a 
range  of  simulated  physiological  input  signals,  and  motivates  refinement  of  the  existing  model  and  extension  to  include 
performance-based  mental  workload  measures,  e.g.,  task  response  time  and  error  data,  as  well.  Operational  evaluation 
in  which  the  network  will  be  driven  by  a  physiological  measurement  system  is  planned  for  the  immediate  future. 

We  provide  an  analysis  that  rank  orders  the  importance  of  the  various  pieces  of  information  available  in  the  cockpit 
(e.g.,  threat  geometry,  type,  and  kinematics,  and  ownship  kinematics  and  weapons)  as  a  function  of  pilot  task, 
situational  state  and  mission  phase.  For  example,  one  important  high-level  measure  of  situational  state  is  the  pilot’s 
overall  ofTensi ve/defensive  posture,  expressible  as  various  “defcon”  levels.  We  then  describe  a  candidate  PV1 
adaptation  strategy,  developed  in  collaboration  with  domain  experts,  which  maps  task/state^hase  configurations  to 
optimal  PV1  configurations. 


(Reprint  of  executive  summary;  formal  paper  not  available.) 
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The  United  States  Air  Force  and  United  States  Navy  are  maturing  advaneed  Real-Time  Information  Into  / 
Out-of  the  Cockpit  (RTIC/RTOC)  technologies  to  enhance  aircrew  situation  awareness,  increase  operational  flexibility 
and  increase  mission  effectiveness  against  time  critical  fixed  and  mobile  targets 

Under  an  umbrella  of  Sensor-To-Shooter  (STS)  System&'Sofhvare  Engineering  and  Integration  activities, 
GDE  Systems,  Inc.  is  supporting  the  development  and  integration  of  onboard'offboard  core  technologies  as  well  as  new 
Concepts  for  Rapid  Targeting  operations.  The  fiscal  year  1 997  STS  efforts  comprise  a  series  of  advanced  technology 
prototype  development  and  demonstration  projects  focused  on  new  oftboard  and  onboard  data  processing  capabilities. 
Initial  proof-of-concept  demonstrations  have  established  feasibility  of  key  components  with  technology  transition 
initiatives  underway. 

Although  STS  objectives  are  oriented  toward  conventional  Global  Reach,  TACAIR  and  Standoff  Weapon 
precision  strike  mission  applications  in  support  of  Joint  Vision  2010,  the  underlying  technology  is  applicable  to 
multiple  mission  areas,  such  as  Army  Deep  Strike,  Naval  Surface  Fire  Support,  Strategic  Nuclear,  Combat  Search  and 
Rescue,  Special  Operations  and  Non-Traditional  Warfare  (e.g.,  crisis  management). 


(Reprint  of  executive  summary;  formal  paper  not  available.) 
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Cue-Recognition  Training  for  Enhancing 
Team  Situational  Awareness 


Renee  J.  Stout,  Eduardo  Salas,  Janis  A.  Can  non- Bowers  and  Eduardo  Salas 
Naval  Air  Warfare  Center  Training  Systems  Division 
Orlando,  FL 

With  the  importance  of  situational  awareness  and  team  situational  awareness  to  tactical  performance  well 
established,  it  is  critical  that  viable  solutions  are  identified  to  enhance  their  development  (Salas,  Prince,  Baker,  & 
Shrestha,  1995;  Stout,  Cannon-Bowers  &  Salas,  1996).  The  key  to  designing  instructional  strategies  to  promote  and 
maintain  situational  awareness  and  team  situational  awareness  relies  upon  understanding  the  process  of  situation 
assessment.  Fortunately,  the  literature  has  provided  some  insight  into  this  process.  That  is,  the  nature  of  expertise  relies 
upon  the  rapid  and  accurate  assessment  of  cues  and  patterns  of  cues  to  build  high  levels  of  situational  awareness  (c.g., 
Cannon-Bowers,  Salas,  &  Grossman,  1991;  Klein,  1989).  Likew  ise,  the  nature  of  team  expertise  embodies  the  ability 
to  pass  and  obtain  relevant  information  among  the  team  that  contributes  to  high  levels  of  team  situational  awareness.  In 
team  settings,  both  of  these  are  required  to  enable  the  taking  of  appropriate  task  actions.  Therefore,  to  expedite  the 
development  of  expertise  and  team  expertise  and  thus  situational  aw  areness  and  team  situational  awareness,  it  is  crucial 
to  help  trainees  to  identify  and  comprehend  relevant  features,  cues,  and  patterns  in  the  task  environment  that  are  often 
quite  subtle. 

Through  a  thorough  review  of  the  literature  across  several  areas  and  interviews  with  operational  military 
pilots,  the  instructional  strategy  of  cue-recognition  training  has  been  identified  as  a  potentially  effective  means  of 
increasing  situational  awareness  and  team  situational  awareness.  Cue-recognition  training  is  referred  to  here  as  the 
process  of  making  relevant  cues  and  patterns  of  cues  more  salient,  thereby  increasing  the  probability  for  which  they  will 
be  attended  Given  the  potential  importance  of  this  instructional  strategy,  the  purpose  of  this  paper  is  threefold-  1)  to 
explicate  the  theoretical  foundation  of  this  instructional  strategy  ;  2)  to  describe  through  example  what  this  instructional 
strategy  might  look  like;  and  3)  to  provide  anecdotal  data  obtained  from  interviews  with  pilots  regarding  the  potential 
utility  of  this  training  approach. 

We  start  by  briefly  summarizing  recent  conceptual  work  that  has  been  done  on  team  situational  awareness 
from  which  the  idea  of  cue-recognition  training  was  initially  derived.  We  turn  next  to  briefly  describing  other  related 
literature  that  supports  the  potential  importance  of  this  instructional  strategy.  Following  this  we  provide  a  more  detailed 
discussion  of  cue-recognition  training  specifically.  Finally,  we  give  examples  from  pilots  regarding  their  reactions  to 
cuc-recognition  training. 

Brief  Summary  of  Team  Situational  Awareness  Literature 

Recent  theoretical  treatments  provided  by  Shrestha,  Prince,  Baker,  and  Salas  (1995),  Salas  et  al.  (1995),  and 
Stout  et  al  (1996)  have  all  discussed  the  criticality  of  team  situational  awareness  and  have  attempted  to  delineate  a 
conceptual  understanding  of  this  construct.  For  example,  both  Shrestha  et  al.  and  Salas  et  al.  offered  the  notion  that 
team  situational  awareness  is  greater  than  the  sum  of  the  situational  awareness  of  individual  team  members,  as  w'ell  as 
stressed  the  importance  of  shared  mental  models  to  the  development  of  team  situational  awareness.  Stout  et  al. 
extended  these  works  by  providing  a  model  of  team  situational  awareness.  In  essence,  their  work  suggests  that  team 
situational  awareness  is  a  dynamically  changing  cognitive  state,  which  is  influenced  by  the  process  of  situation 
assessment  This  process  of  situation  assessment  is  in  turn  influenced  by;  I )  mental  models  and  shared  mental  models 
that  team  members  bring  to  bear  to  the  task  situation;  2)  cues/pattems  of  cues  that  occur  in  the  dynamic  task  situation, 
which  come  from  a  variety  of  sources,  and  3)  team  process  behaviors,  which  influence  both  individual  models  of  the 
situation,  or  individual  situational  awareness,  and  team  level  situational  awareness.  According  to  these  authors,  team 
situational  awareness  is  high  when  the  situation  has  been  assessed  compatibly  by  the  team  members,  enabling  them  to 
work  in  concert 

The  Stout  et  al.  framework  yields  implications  for  training  team  situational  awareness.  That  is,  given  that  it  is 
a  dynamically  changing  cognitive  state,  we  cannot  actually  train  team  situational  awareness  per  se,  but  we  can  train  the 
process  of  situation  assessment,  one  critical  component  of  which  is  making  trainees  aware  of  the  relevant  cue/pattems 
in  their  specific  task  situation,  Given  the  potential  positive  impact  that  accurate  and  appropriate  cue/pattem  assessments 
can  have  on  developing  high  levels  of  team  situational  awareness,  we  focus  here  on  a  strategy  that  may  improve 
cue/pattem  assessments  -  cue-recognition  training  -  prompting  the  trainees’  attention  to  relevant  cues/pattems  of  cues. 
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A  vast  body  of  literature  across  several  domains,  such  as  cognitive  and  educational  psychology  also  supports  the 
potential  effectiveness  of  this  strategy.  While  a  review  of  this  literature  is  beyond  the  scope  of  this  paper,  we  briefly 
review  some  particularly  relevant  research  in  the  next  section. 

Support  for  the  Link  Between  Oie/Pattern  Assessments  and  Human  Performance 

Relevant  features,  cues,  and  patterns  of  cues  in  a  particular  task  context  are  often  quite  subtle.  Recognizing 
and  assessing  more  quickly  and  completely  these  aspects  of  the  task  have  often  been  cited  as  a  key  to  developing 
domain  expertise  (e.g„  Druckman  &  Bjork,  1994;  Means,  Salas,  Crandall,  &  Jacobs,  1993).  A  set  of  studies  conducted 
by  Bransford  and  colleagues  and  summarized  by  Bransford,  Franks,  Vye  and  Sherwood  (1989)  serves  as  an  excellent 
example  of  bodies  of  research  that  have  concentrated  on  determining  how  individuals  come  to  discriminate  one  case 
from  another.  In  this  work  on  cue  learning,  Bransford  et  al.  explained  that  individuals  must  leam  the  process  rather 
than  the  outcome  of  cue  assessments.  For  example,  pointing  out  to  the  trainee  that  “the  territory  ahead  is  hostile"  is  the 
output  of  an  expert’s  assessment  and  does  not  allow  the  trainee  to  leam  the  underlying  cues  that  led  to  this 
determination.  According  to  these  authors,  it  is  far  more  important  to  specify  what  the  relevant  cues  arc  that  allow  one 
to  conclude  that  a  territory  is  hostile  and  to  compare  and  contrast  these  cues  under  differing  conditions.  For  example, 
indicating  that  an  aircraft  is  an  enemy  because  of  critical  features  “1,2,  and  3,"  when  it  is  in  close  proximity  to  another 
aircraft  with  both  similar  and  different  features,  provides  relevant  contrasts  and  allows  cues  to  take  on  meaning. 

In  addition  to  more  basic  research  on  cue  learning,  a  body  of  applied  work  has  amassed  which  advocates  the 
importance  of  rapid  and  accurate  cue  pattern  assessments  to  effective  decision  making  and  performance  (Adams, 
Tenney,  &  Pew,  1995;  Cannon-Bowers  &  Bell,  1997;  Endsley,  1995a,  1995b;  Gaba,  Howard,  &  Small,  1995;  Kaempf 
&  Klein,  1994;  Kaempf,  Wolf,  Thordsen,  &  Klein,  1992,  Klein,  1989;  Orasanu,  1990;  Salas  et  al.,  1995;  Sarter  & 
Woods,  1991;  Stout  et  al.,  1996).  Indeed,  in  Klein’s  well-known  model  of  recognition  primed  decision  making  (1989), 
perhaps  the  most  important  factor  in  effective  decision  making  among  experts  is  to  correctly  assess  the  situation. 
According  to  Klein,  experts  often  make  correct  choices  and  take  appropriate  action  in  complex,  cue-rich  contexts  by 
using  their  experience  to  draw  accurate  inferences  from  available  cues  and  patterns. 

Studies  that  have  directly  examined  how  experts  and  novices  use  cues  also  provide  insight  into  the  importance 
of  assessing  cues  and  patterns  in  task  accomplishment  (e.g.,  Myles-Worsley,  Johnston,  &  Simons,  1988;  Kaempf  & 
Klein,  1994).  In  general,  these  studies  lend  support  that  experts  are  often  unable  to  articulate  the  specific  cues  that  they 
attended  to  and/or  how  they  used  these  cues  to  make  their  assessments.  Also,  surprisingly,  experts  often  use 
unexpected  strategies  in  assessing  cues  and  patterns. 

In  summary,  the  efforts  focused  on  team  situational  awareness  and  those  described  in  this  section  are 
important,  because  they  theoretically  support  the  potential  application  of  a  strategy  aimed  at  improving  cue/pattem 
assessments.  The  latter  research  on  expert  and  novice  differences  is  also  particularly  important,  because  to  use  strategy 
to  improve  team  situational  awareness  that  enhances  cue 'pattern  assessment,  it  is  obviously  crucial  to  determine  which 
cues  to  point  out  to  the  trainee.  Because  it  is  difficult  to  gather  this  information  from  experts  (given  conscious 
u n awareness),  a  paradox  arises.  Fortunately,  the  area  of  cognitive  engineering  has  y  ielded  techniques  that  can  be  used 
to  identify  training  content  by  “teasing  out"  this  information  (see  Cooke,  1994,  for  a  review  of  these  techniques). 
Clearly,  it  would  be  important  to  utilize  these  techniques  when  developing  the  content  of  cue-recognition  training. 
Given  a  better  understanding  of  what  to  cue ,  we  now  turn  to  how  cue-recognition  training  should  take  place. 

How  Might  We  Cue? 

Attaining  and  sustaining  a  high  level  of  situational  awareness  is  primarily  based  upon  the  accurate  assessment 
of  relevant  cues/pattems  of  cues  in  any  given  situation  (Adams  et  al.,  1995;  Endsley,  1995a;  Salas  et  al.,  1995;  Stout  et 
al.,  1996).  Actively  prompting  individuals  or  teams  to  take  note  of  cues  that  are  pertinent  to  successfully  completing 
their  tasks  is  one  possible  training  strategy  for  elevating  the  level  of  situational  awareness  attained.  As  mentioned,  we 
term  this  training  process  of  guiding  individuals  to  attend  to  relevant  task  parameters  as  cue-recognition  training. 
Taking  into  consideration  each  of  the  research  examples  outlined  in  the  prior  section,  we  now  endeavor  to  explain  how 
cue-recognition  training  could  be  made  possible.  We  suggest  that  in  each  specific  case,  cue-recognition  training  would 
be  both  appropriate  and  potentially  beneficial 

First,  Bransford  et  al.  (1989)  contended  that  it  is  imperative  to  guide  trainees,  from  the  beginning,  to  take 
notice  of  crucial  features  as  they  are  learning  new  information  They  also  offered  that  experts  have  a  wealth  of  diverse 
experiences  to  draw  upon  when  presented  with  a  set  of  stimuli.  This  provides  them  with  internal  contexts  and 
alternatives,  which  may  not  be  readily  apparent  to  the  novice  Mann  and  Decker  (1984)  supported  this  notion  in  that 
they  described  a  cue’s  distinctiveness  (i.e.,  how  it  contrasts  with  other  cues)  and  meaningfulness  as  critical  to  the 
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learning  process.  Therefore,  it  may  be  suggested  that  presenting  novices  with  relevant  contrasts  in  respective 
environments  causes  germane  cues  to  become  more  apparent  and  thus  better  learned. 

Based  upon  the  above  assumption,  scenarios  could  be  designed  with  a  myriad  of  system  malfunctions,  for 
example,  in  which  cues  may  or  may  not  be  relevant  to  the  malfunction  An  instructor  could  then  point  out  the  cues  that 
are  relevant  to  the  malfunction  This  w  ould  aid  the  trainee  in  developing  an  a.euute  mental  model  of  the  pertinent  cues 
to  look  for  when  a  part  of  the  system  is  malfunctioning.  Additionally,  the  trainee  would  become  aware  of  which 
cues/pattems  of  cues  were  not  predictive  of  system  performance  and  thus  not  important  to  attend  to.  Distinctiveness  is 
the  key  to  making  cues  relevant  to  the  novice  (Bransford  et  al.,  1989;  Mann  &  Decker,  1984),  This  can  be 
accomplished  by  displaying  the  cue 'behavior  out  of  context,  exaggerating  the  behavior,  repeating  the  behavior 
frequently  and  by  using  learning  points  to  identify  the  crucial  determinants  of  the  behavior.  Several  methods  may  attain 
the  goal  of  creating  contrastive  learning  effects;  Bransford  et  al.  suggested  the  use  of  random  access  videotape  to  show 
the  trainee  contrasting  segments. 

To  date,  studies  on  situation  assessment,  that  have  investigated  the  cues  that  experts  attend  to,  posit  that  it  is 
critical  that  trainees  have  awareness  of  cues  that  are  pertinent  to  their  task  performance.  Specifically,  trainees  could 
potentially  learn  an  inaccurate  set  of  cues  w  hich  w  ould  result  in  a  flawed  mental  model.  In  a  time  compressed  situation, 
this  could  be  fatal.  Accurate  mental  models/shared  mental  models  are  imperative  to  high  levels  of  team  situational 
awareness  (Stout  et  al.,  1996)  therefore,  we  suggest  that  cue-recognition  training  can  foster  the  development  of  team 
situational  awareness.  In  the  next  section,  we  delineate  a  strategy  for  implementing  cue-recognition  training. 
Additionally,  we  describe  in  greater  depth  how  this  particular  strategy  could  increase  team  situational  awareness 
thereby  enhancing  performance, 

How  Might  VVc  Implement  Cue-Recognition  Training 

First,  there  are  a  number  of  ways  that  cue-recognition  training  can  be  delivered.  Cue-recognition  training  can 
take  place  through  passive  demonstrations  or  while  the  trainee  is  practicing  his/her  tasks.  It  can  also  occur  through 
instructor  input  or  it  can  be  delivered  by  the  system  itself  (such  as  through  highlighting  relevant  gauge  cues). 
Additionally,  information  can  be  presented  directly  or  through  questioning  the  trainees  about  what  cues  are  relevant 
given  the  situation.  The  cues  presented  would  come  from  a  variety  of  operational  sources,  such  as  the  extemal/intemal 
environment,  the  mission,  the  cockpit  instrumentation,  and  teammates  In  addition  to  prompting  the  trainees’  attention 
to  relevant  cues  and  patterns  of  cues,  instruction  could  be  provided  regarding  what  the  trainee  should  “say”  and  “do” 
based  upon  the  situation  at  hand. 

Four  subsets  of  cue-recognition  training  are  proposed,  based  upon  a  review  of  the  literature  which  has 
described  strategies  related  to  some  type  of  guidance  or  prompting  (summarized  in  Figure  1 ); 

CUE-RECOGNITION  TRAINING 


System 


Instructor 


Passive  Active 


Passive  System  Prompting 

Active  System  Prompting 

Behavioral  Coaching 

Instructor-Guided  Practice 

Figure  1.  Categorization  of  cue-recognition  training  methods. 


Note.  From  “A  team  perspective  on  situational  awareness  (SA):  Cueing  training”  by 
Stout,  R  J.,  Cannon- Bowers,  J  A.,  &  Salas,  E.  (1997).  [CD-ROM].  Proceedings  of  the 
1 9lh  annual  l  nterserviceJ  Industry  Training,  Simulation  and  Education  Conjerence, 

Orlando,  FL,  174-182.  Reprinted  by  permission. 

Passive  System  Prompting:  cue-recognition  training  which  occurs  through  a  passive  demonstration,  such  as  a 
videotape  or  a  static  system  demonstration,  which  shows  the  system  prompting  the  relevant  information  as  in 
active  system  prompting. 

Active  System  Prompting:  cue-recognition  training  which  occurs  on-line,  or  as  the  individual/team  is 
practicing  their  tasks  and  which  is  provided  by  the  system  (e.g.,  via  highlighting  or  fading). 
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Behavioral  Coaching:  cue-recognition  training  which  occurs  through  a  passive  demonstration,  such  as  a 
videotape  or  a  real  time  demonstration  by  an  instructor  in  which  the  instructor  verbalizes  the  cues  he/she  is 
attending  to  in  accomplishing  the  tasks,  the  relevant  processes,  and  the  necessary  steps  being  taken 
Instructor-Guided  Practice;  cue-recognition  training  which  occurs  on-line,  or  as  the  individual/team  is 
practicing  their  tasks  and  which  is  provided  via  instructor  comments,  where  the  instructor  points  out  the  cues 
to  attend  to,  the  processes  to  attend  to,  and  the  necessary  steps  to  take  to  accomplish  the  tasks. 

In  each  of  these  cases,  the  information  can  be  either  directly  presented  as  statements  of  fact,  can  be  presented  by 
questioning  the  trainee,  or  can  use  a  combination  of  the  methods 

These  four  teaching  methods  can  all  be  used  to  improve  the  level  of  team  situational  awareness  achieved  In 
all  four  methods,  this  can  be  accomplished  when  the  relevant  cues/pattems  of  cues  are  pointed  out  so  that  an  accurate 
system  assessment  can  be  made.  In  addition  to  pointing  out  the  relevant  cues  to  attend  to,  emphasis  would  be  placed  on 
appropriate  team  processes  to  engage  in  based  upon  the  particular  situation  assessment.  All  four  of  these  strategies  can 
aid  in  the  understanding  of  specific  situations  resulting  in  compatible  mental  models  among  teammates.  These 
strategies  can  supply  trainees  with  information  relevant  to  evolving  task  conditions  in  order  to  provide  contrasts.  As  a 
result  of  these  contrasts,  trainees  will  know'  w  hich  information  to  share,  will  interpret  cues  consistently  and  will  share  all 
of  this  information  in  an  expected  manner  with  fellow  teammates.  While  the  passive  strategies  also  impart  strategic 
knowledge  to  the  trainees,  the  training,  in  these  two  cases,  is  not  as  complete  due  to  the  passive  nature  inherent  in  these 
two  techniques  The  active  training  strategies  that  provide  a  more  “hands-on”  approach  are  better  suited  for  increasing 
strategic  knowledge  and  hence  effect  team  situational  awareness.  The  choice  of  one  method  over  another,  however,  is 
probably  best  made  on  practical  grounds  based  upon  available  resources. 

Research  is  needed  to  test  each  of  these  training  strategies  to  determine  their  impact  on  team  situational 
awareness.  With  the  active  cue-recognition  training  strategies  (i  e ,  instructor-guided  practice  and  active  system 
prompting),  we  hypothesize  that  individual  team  members  will'  a)  fonn  expected  and  appropriate  task  strategies;  b)  be 
able  to  accurately  predict  a  team  member’s  behavior;  c)  develop  common  expectations  of  additional  task  and 
information  requirements;  d)  form  common  explanations  of  the  meaning  of  task  cues;  and  e)  form  compatible  situation 
assessments.  Research  is  also  needed  to  determine  the  sequencing  that  these  strategies  should  follow.  For  example,  the 
passive  strategies  may  need  to  precede  the  active  strategies  to  maximize  the  effect  on  team  situational  awareness. 
Additionally,  it  is  crucial  that  research  be  directed  at  determining  the  pertinent  cues  that  should  be  emphasized  during 
the  course  of  training.  This  can  only  be  accomplished  successfully  by  employing  a  systematic  framework  to  guide  the 
research  and  by  using  emerging  techniques,  such  as  knowledge  elicitation  approaches  We  suggest  that  the  Stout  et  al. 
model  can  be  used  as  the  cadre  to  guide  this  research.  Next,  an  example  is  provided  to  illustrate  what  a  cue-recognition 
training  method  might  look  like  and  how  it  could  potentially  aid  in  enhancing  team  situational  awareness. 

What  Might  a  Cue-recognition  Training  Approach  Look  Like? 

We  provide  a  military  transport  helicopter  Search  and  Rescue  (SAR)  mission  example  to  illustrate  what 
a  cue-recognition  training  approach  might  look  like.  The  crew  complement  in  this  aircraft  is  that  of  a  pilot,  a 
copilot,  and  crew  chiefs  who  operate  in  the  rear  of  the  aircraft.  Let  us  assume  that  the  downed  aircraft  is  in  the 
water,  so  the  SAR  helicopter  is  in  a  high  hover  sending  the  swimmer  down  to  retrieve  survivors.  When  the 
swimmer  has  been  lifted  half  way  back  up  the  hoist  with  one  survivor,  suddenly  the  aircraft  experiences  an 
engine  malfunction.  Let  us  also  assume  that  knowledge  elicitation  techniques  from  cognitive  engineering  were 
employed  to  determine  from  subject  matter  experts  the  relevant  cues  in  this  situation.  In  this  case,  let  us  say  that 
experts  tended  to  agree  that  the  most  important  cues  related  to  correctly  diagnosing  the  engine  problem, 
maintaining  safety  of  flight,  and  correctly  deciding  whether  to  put  the  helicopter  in  the  water  and  when  and 
where  to  do  so.  Assume  that  the  experts  agreed  that  engine  diagnostic  cues  are  from  engine  gauges  (e.g.,  torque, 
rpm,  NF  relative  to  NR,  temperature,  and  whether  or  not  an  engine  chip  light  was  illuminated).  Assume  also 
that  safety  of  flight  cues  are  from  both  cockpit  instrumentation  (e.g.,  airspeed  and  altitude)  and  from  the 
environment  (e.g.,  visual  cues  from  the  water  indicating  the  helicopter’s  position  in  space).  Finally,  assume  that 
cues  related  to  taking  action  involve  knowing  where  the  swimmer  is.  At  times  this  information  can  only  be 
obtained  via  crew  chief  communications. 

Given  all  of  this  information  as  a  baseline,  a  cue-recognition  training  approach  should  be  geared  toward 
imparting  to  the  trainee  that  each  of  these  cues  is  relevant,  as  well  as  toward  making  the  trainee  aware  of  when 
these  cues  are  important.  We  would  also  want  them  to  understand  the  relative  significance  of  each  of  the  cues. 
In  addition,  it  would  be  important  for  trainees  to  know  what  cue  information  must  be  communicated  to  other 
team  members  to  build  a  compatible  understanding  of  the  situation  and  to  thereby  allow  appropriate  coordinated 
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action  to  be  taken.  Using  our  notion  of  behavioral  coaching  as  an  example,  an  instructor  could  prompt  the 
trainee’s  attention  to  each  of  these  issues,  either  while  performing  in  the  simulator  or  by  using  a  videotape  of  the 
simulation  from  which  to  discuss  the  situation.  That  is,  the  instructor  could  explain  to  the  trainee  what  it  is  that 
he/she  would  be  thinking  about  and  saying  as  each  member  of  the  crew.  This  could  include  detailing  what 
information  should  be  passed  to  other  crewmembers  and  stating  the  problem  in  a  manner  that  allows  all 
crewmembers  to  understand  what  is  happening.  The  instructor  would  be  guided  in  his/her  behavioral  coaching 
technique  to  utilize  the  information  gathered  through  the  knowledge  elicitation  methods.  The  following 
example  shows  what  an  instructor  may,  therefore,  say. 

"As  a  pilot  or  copilot,  given  that  we  are  in  a  high  hover  over  water,  1  might  expect  salt  from  the  ocean  to 
cause  some  type  of  problem  with  the  engine,  so  1  would  be  scanning  my  engine  gauges  As  a  pilot,  1  would 
also  continue  to  scan  altitude  and  airspeed  to  maintain  a  safe  hover,  and  1  would  keep  an  eye  on  the  water  to 
see  if  the  waves  look  larger  or  smaller,  and  to  see  if  1  am  level.  As  a  copilot,  I  would  back  the  flying  pilot 
up  by  looking  at  the  same  cues.  The  first  indication  of  an  engine  problem  would  probably  be  a  change  in 
torque  or  rpm  Either  the  pilot  or  copilot  might  notice  it,  and  then  each  of  us  would  look  for  secondaries, 
expecting  either  NF  to  go  below  NR  (or  for  NF  to  split  off  from  NR)  or  a  change  in  engine  temperature.  If 
either  or  both  of  these  occur  we  might  ask  the  crew  chief  to  look  for  any  other  secondaries  in  the  back,  but  1 
would  assume  that  we  have  an  engine  problem  and  would  ask  my  copilot  to  execute  engine  shut  down 
procedures  while  1  watch  for  safety  of  flight  items  (altitude,  airspeed,  and  visual  water  cues).  Given  that  we 
are  over  water  with  an  engine  problem  and  are  not  single-engine  capable,  we  may  need  to  land  it  in  the 
water  as  soon  as  possible,  so  1  would  have  stated  the  symptoms  that  I  had  seen  originally  to  all  crew  and 
stated  that  there  was  a  possible  engine  problem,  and  once  confirmed  by  the  secondaries,  1  would  tell  the 
crew  that  we  need  to  land  it  in  the  water  as  soon  as  possible.  The  crew  chief  would  then  look  for  the 
swimmer,  if  he/she  was  still  on  the  hoist,  and  make  a  determination  of  whether  or  not  to  cut  the  sw  immer  or 
bring  him/her  on  board  based  upon  his/her  distance  from  the  water,  which  would  be  communicated  to  me. 

Once  safe,  I  would  tell  the  crew  that  we  are  putting  it  in  the  water  The  crew  chief  would  let  me  know  that 
the  passengers  are  strapped  in  and  I  would  land  it  in  the  water.” 

The  key  distinction,  between  the  approach  that  we  offer  and  traditional  behavioral  coaching  approaches  is  that  the 
instructor  is  emphasizing  critical  cues,  and  what  he/she  is  thinking  and  communicating  to  develop  and  maintain  team 
situational  awareness  rather  than  simply  stating  what  he/she  is  doing.  Keeping  in  line  with  what  was  suggested  by 
Bransford  et  al,  (1989),  the  output  of  the  expert's  judgment  is  not  what  is  presented  -  rather,  the  processes  used  to  make 
that  judgment  are  presented. 

This  section  has  simply  served  as  one  example  of  the  myriad  of  specific  instruction  that  could  be  provided  via 
cue-recognition  training.  Again,  research  is  greatly  needed  to  empirically  determine  how  best  to  implement  this 
instruction. 

Given  a  better  understanding  of  what  we  mean  by  cue-recognition  training  and  that  the  literature  review  ed  in 
this  paper  supplied  hypothetical  evidence  of  the  potential  usefulness  of  this  training  approach,  we  next  sought  reactions 
about  this  technique  from  operational  personnel.  We  describe  some  of  these  reactions  next. 

Fleet  Reactions  to  Cue-Recognition  Training 

Our  approach  to  determining  what  aviators  think  about  the  potential  effectiveness  of  a  cue-recognition 
training  scheme  was  to  first  discuss  with  them  our  notion  of  team  situational  awareness  and  to  then  provide  examples  of 
what  this  training  might  look  like  in  a  manner  consistent  with  that  presented  in  this  paper.  We  interviewed  47  aviators 
from  5  different  operational  communities.  A  very  typical  response  obtained  from  the  aviators  was  that  they  did  not  feel 
that  team  situational  awareness  could  be  trained.  That  is,  they  felt  that  “experience”  is  what  is  needed.  After  careful 
explanation  of  our  ideas,  however,  many  aviators  commented  that,  w  hile  they  had  not  received  a  formal  systematic  cue- 
recognition  training  during  their  careers,  “good  instructors  informally  provide  some  form  of  cue-recognition  training  ... 
such  as  saying,  hey  get  your  head  inside  the  cockpit  now  and  look  at  this  gauge  because...’’  A  typical  response  was 
also  to  the  effect  that  “while  good  instructors  do  this  kind  of  thing,  I  have  often  wondered  if  what  they  are  saying  is 
correct,  because  it  is  just  one  instructor’s  opinion,  and  a  technique  that  makes  this  more  systematic  and  pools  opinions 
would  give  me  greater  confidence  in  the  product  ”  In  addition,  almost  universally,  pilots  commented  that  this  approach 
would  be  particularly  valuable  for  student  pilots.  Finally,  pilots  noted  the  importance  of  providing  active  practice 
opportunities  with  cue-rich  scenarios.  Thus,  initial  reactions  of  pilots  toward  cue-recognition  training  were  favorable. 
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CONCLUSIONS 


The  purpose  of  this  paper  was  to  extend  our  understanding  of  how  team  situational  awareness  can  be 
improved  through  training.  It  did  so  by  elaborating  upon  one  potential  training  strategy  derived  from  the  literature  and 
positively  reacted  to  by  aviators  -  cue-recognition  training.  This  is  important  because  it  is  a  first  step  in  linking  theory 
to  practice  in  this  area.  Obviously,  research  must  be  conducted  to  empirically  validate  cue-recognition  training.  We 
hope  that  our  paper  stimulates  interest  and  research  in  this  important  arena. 
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Application  of  the  Virtual  Retinal  Display™  (VRD™)  to  the 
Virtual  Cockpit  Optimization  Program  (VCOP) 


Thomas  M.  Lippert 

Microvision,  Inc. 

Seattle,  WA 

In  the  last  decade,  the  Army  has  successfully  developed  technologies  designed  to  improve  the  rotary¬ 
wing  cockpit  environment  and  individual  aircrew  equipment  for  tactical  flight  operations.  Since  many  of  these 
technologies  have  now  matured,  the  Army  has  recognized  the  need  to  pool  various  research  and  development 
results  to  determine  which  technologies  should  be  further  explored  to  optimize  pilot  performance  in  current  and 
future  aircraft  cockpits.  The  Army's  VCOP  initiative  will  analyze  the  Pilot  Vehicle  Interface  (PV1)  and 
recommend  insertion  of  the  best  technologies  into  the  existing  helicopter  fleet. 

There  is  a  rapidly  accelerating  need  to  control  and  manage  information  currently  provided  to  the  Army 
Aviator  in  order  to  optimize  his  ability  to  perform  effectively  in  the  cockpit  and  on  the  digitized  battlefield.  To 
date,  the  Army  has  amassed  an  arsenal  of  tools  to  assist  the  pilot  functioning  in  the  cockpit,  VCOP  has  been 
formalized  to  identify  and  select  optimal  cockpit  technologies  for  actual  integration  in  today’s  aircraft.  This 
program  will  evaluate  off-the-shelf  and  emerging  technologies  from  all  military  departments  which  could  best 
apply  to  the  unique  demand  of  the  highly  complex  helicopter  cockpit  and  pilot  situational  awareness 
environment.  At  the  conclusion,  the  best  of  the  Army’s  recent  R&D  initiatives  can  be  properly  identified  and 
applied  to  the  current  helicopter  inventory. 

Current  plans  call  for  the  Virtual  Cockpit  Optimization  Program  to  evaluate  a  full-color,  high 
resolution,  high  brightness  helmet-mounted  display,  three-dimensional  audio,  tactile  display,  voice  recognition, 
and  voice  synthesis.  The  resultant  all-electronic  crew  station  will  enable  the  evaluation  of  a  simulated, 
reconfigurable  cockpit  in  which  the  pilots’  abilities  to  interface  with  both  aircraft  and  environment  can  be 
optimized  independent  of  external  conditions  such  as  degraded  visibility.  Such  an  evaluation  capability  can  also 
be  viewed  as  an  important  step  toward  remote  pilotage  of  high  performance  aircraft.  Simulation  capabilities  will 
be  specifically  employed  to  validate  selected  technologies  and  their  potential  for  integration  with  existing 
systems.  VCOP  success  could  readily  enhance  improvement  in  similar  cockpit  environments  in  armor  and  self- 
propelled  artillery  systems. 

Because  pilot  situational  awareness  is  key  to  mission  success,  the  majority  of  imagery  and  information 
needs  to  be  presented  in  an  out-the-window  format,  and  in  a  manner  that  the  pilot  will  readily  comprehend 
VRD  provides  the  necessary  brightness,  contrast,  and  resolution  for  both  day  and  night  operations  for  VCOP 
mission  requirements.  Unlike  other  display  technologies  such  as  flat  panel  displays,  the  VRD  is  an  integrated 
display  system  with  modular  components,  which  can  be  developed  in  parallel  instead  of  sequentially.  This 
modularity  permits  rapid  advancement  of  and  improvements  to  system  performance,  while  at  the  same  time 
continuing  to  decrease  overall  VRD  system  size. 


(Reprint  of  executive  summary;  formal  paper  not  available.) 
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Flight  Simulator  Comparison  of  Two  Tactical  Cockpit  Configurations 
Under  Static  and  High-G  Conditions 


Harold  D.  Warner  and  Norman  Warner 

Naval  Air  Warfare  Center  Aircraft  Division 
Patuxent  River,  MD 

A  two-phase  flight  simulation  research  program  was  conducted  to  compare  two  tactical  cockpit  configurations 
in  relation  to  tactical  mission  performance.  One  of  the  configurations,  defined  as  the  advanced  configuration, 
incorporated  advanced  control  and  display  concepts  and  weapon  capabilities.  The  other,  called  the  standard 
configuration,  employed  conventional  cockpit  and  weapon  technologies  These  cockpit  configurations  were  compared 
under  static  conditions  in  the  first  phase  of  the  research  program  and  under  high  vertical  gravitational  (Gz)  conditions  in 
the  second  phase. 

The  reconfigurable  cockpit  developed  by  the  Boeing  Company  as  an  advanced  concept  demonstrator  was 
used  to  provide  the  two  cockpit  configurations.  The  cockpit  is  installed  in  the  gondola  of  the  centrifuge  located  at  the 
Veda,  Inc.,  facility  in  Warminster,  PA.  It  is  equipped  with  a  forward  instrument  panel  that  contains  four  multifunction 
displays  (MFDs),  three  cathode  ray  tube  (CRT)  monitors  for  displaying  the  computer-generated  outside  visual  scenes 
and  head-up  display  (HUD)  symbology,  and  a  variety  of  operational  controls.  The  cockpit  controls  include  a  side-arm 
control  stick,  a  throttle,  a  master  mode  switch,  and  a  master  arm  switch,  Various  switches  are  integrated  into  both  the 
control  stick  and  throttle  to  provide  hands-on-stick-and-throttle  (HOTAS)  switch  functionality. 

In  the  advanced  cockpit  configuration,  the  simulated  aircraft  is  loaded  with  two  Joint  Direct  Attack  Munitions 
(JDAMs)  and  four  AIM- 120  missiles.  The  JDAM  is  a  global  positioning  system  (GPS)-guided,  “smart”  bomb  that 
autonomously  steers  to  the  target  designated  by  the  pilot,  and  is  released  at  high  altitude  to  reduce  aircraft  exposure  to 
enemy  defenses.  The  JDAMs  are  delivered  using  the  target  attack  MFD,  which  displays  ground  target  and  JDAM 
impact  envelope  symbology.  Deployment  of  the  missiles  is  accomplished  using  air-to-air  weapon  symbology 
displayed  on  the  HUD.  In  the  standard  configuration,  the  aircraft  has  two  conventional  Mk-82  bombs  and  also  four 
AIM-120  missiles.  Continuously  computed  impact  point  (CCIP)  symbology  displayed  on  the  HUD  is  used  to  deliver 
the  Mk-82  bombs.  The  bombs  are  delivered  at  low  altitude,  and  the  CCIP  symbology  must  be  precisely  positioned 
over  the  target  in  the  visual  scene  for  the  weapon  to  hit  the  target.  As  with  the  advanced  cockpit  configuration,  the 
missiles  are  fired  using  the  air-to-air  weapon  symbology  displayed  on  the  HUD. 

The  simulated  tactical  mission  consisted  of  three  segments:  terrain  following,  air-to-ground  weapon  deliveries, 
and  air-to-air  combat.  The  terrain-following  segment  involved  flight  over  mountinous  terrain  and  required  sharp  and 
left  and  right  turns  to  induce  high  G  forces.  There  were  two  ground  targets  and  four  air  targets.  The  ground  targets 
were  a  large  canister,  representing  an  oil  storage  tank,  and  a  bridge;  the  air  targets  were  four  moving  MiG-29  models. 
The  performance  measures  used  in  both  study  phases  were  (1)  time  between  mission  start  and  cockpit  air-to-ground 
mode  switch  activation,  (2)  time  between  cockpit  air-to-ground  mode  switch  activation  and  first  bomb  release,  (3)  time 
between  bomb  releases,  (4)  time  between  cockpit  air-to-air  mode  switch  activation  and  first  missile  release,  (5)  time 
between  first  and  last  missile  releases,  (6)  number  of  ground  targets  destroyed,  (7)  number  of  air  targets  destroyed,  and 
(8)  NASA-TLX  workload  ratings. 

There  were  four  participants  in  the  first  phase.  Each  performed  the  mission  twice  in  both  cockpit 
configurations  under  static,  no-motion  conditions.  Analysis  of  variance  (ANOVA)  statistical  tests  showed  that  there 
were  no  significant  differences  in  performance  between  cockpit  configurations  and  repetitions,  and  that  there  were  no 
significant  interactions  between  cockpits  and  repetitions  for  any  of  the  performance  measures. 

Six  participants  were  used  in  the  second  phase,  and  each  also  performed  the  mission  twice  in  the  two  cockpit 
configurations.  The  centrifuge  was  operational  during  the  flights,  and  the  participants  were  subjected  to  a  maximum  of 
7.0Gz.  ANOVA  results  indicated  that  there  were  significant  differences  between  the  two  cockpit  configurations  for  (1 ) 
time  between  cockpit  air-to-ground  mode  switch  activation  and  first  bomb  release,  (2)  time  between  bomb  releases,  (3) 
number  of  ground  targets  destroyed,  and  (4)  four  of  the  six  workload  scales.  In  these  analyses,  performance  was  better 
and  workload  was  less  in  the  advanced  cockpit  than  in  the  standard  cockpit  configuration  Only  the  “effort”  workload 
ratings  were  significantly  different  between  repetitions,  and  none  of  the  cockpits  by  repetitions  interactions  were 
significant.  In  conclusion,  this  research  demonstrated  that  the  advanced  cockpit  technologies  facilitated  tactical  mission 
performance,  but  the  performance  benefits  were  not  evident  until  the  participants  were  subjected  to  realistic  Gz 
conditions. 

(Reprint  of  executive  summary;  formal  paper  not  available.) 
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The  Effects  of  3D  Audio  on  Tactical  Situation  Awareness 


Valerie  Gawron 

Calspan,  an  Operation  of  Veridian 
Buffalo,  New  York 

Introduction 

Three-dimensional  (3D)  audio  uses  the  cues  enabling  humans  to  localize  sounds  in  their  natural 
environments  to  create  the  perception  of  a  sound  in  space  Theses  cues  are  described  in  greater  detail  below. 
Binaural  Cues 

Position  of  a  sound  can  be  determined  using  binaural  cues,  specifically  sound  reaching  the  ears  at 
different  times  (as  much  at  700  psec)  and  intensities  (as  much  as  40  dB).  Localization  of  sounds  below  2000 
Hz  is  based  primarily  on  time  differences,  above  4000  Hz  on  intensity  differences.  Localization  is  poorest 
between  2000  and  4000  Hz  (Figure  1 ). 


Frequency 

(a) 

Figure  1 .  20-Hz  to  4-kHz  frequency  response  (Begault,  1 901 ,  p.  869) 

Sounds  20  or  1 20  degrees  straight  ahead  have  the  greatest  intensity  difference  (Figure  2). 
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Figure  2.  Errors  in  headphone  localization  for  left  and  right  60  and  1 20° 
target  azimuths  at  0°  elevation  (Begault,  1 991 ,  p.  867). 


Sound  intensity  is  the  primary  cue  for  distance  to  a  sound  source. 

Phase  Cues 

Phase  cues  are  also  used  for  localization  of  periodic  sounds  but  only  if  successive  cycles  are  at  least 
1600  Hz  apart.  Rise  times  of  100  msec  or  greater  also  aid  in  sound  localization.  Distance  and  elevation  also 
have  effects  (Figures  3a  and  b,  respectively). 


43 


Oetince 


90 


Elevaii 


•30  , 
i 

-flOi 


T- 

i 

,■  i 

if 

*10  *13  *1«  *3 


*6  *7  *e  *9 


rt»*o= 

514 
SO  = 
22  1 


iuCurct  \ 


(a) 


subject  * 


Figure  3.  Summary  of  distance  and  elevation  judgments. 

(a)  Targets  at  0°  azimuth,  0°  elevation. 

(b)  Targets  at  180°  azimuth,  0°  elevation  (Begault,  1991.  p.  868) 


Additional  Cues 

Head  movements  provide  cues  but  only  for  sounds  with  durations  greater  the  250  msec  Monaural 
cues  (sound  shadowing  by  the  head  and  loudness  changes  of  moving  sounds)  are  also  used  in  sound 
localization  Localization  can  also  be  affected  by  memory  (where  your  phone  is)  and  cognitive  cues  (where 
speech  emanates  from  a  speaker).  “Environmental  cues  include  the  effects  that  the  listening  environment 
imposes  on  distant  sources,  early  reflections  from  the  ground,  walls  and  ceilings,  and  reverberation,  or  late 
reflections.”  (Scarborough,  1992,  p.  2). 


3D  Technology 


3D  systems  are  being  produced  by  many  companies  (Table  I).  With  competition  has  come 
improvements  in  the  technology.  These  improvements  include  binaural  recording,  “a  method  of  making 
recordings  that  capture  the  temporal  and  spectral  cues  that  exist  in  binaural  sound”  (Scarborough,  1992,  p.  3) 
and  the  use  of  head-related  transfer  function  (HRTF).  “The  HRTF  imposes  a  unique  frequency  response  for  a 
given  sound-source  position  outside  of  the  head,  which  can  be  measured  by  recording  the  impulse  response  in 
or  at  the  ear  canal  and  then  examining  its  frequency  response  using  fast  Fourier  Transform  techniques.  The 
binaural  impulse  response  can  also  be  directly  implemented  into  a  pair  of  digital  filters  for  use  in  a  3D  audio 
system,  using  convolution  techniques”  (Begault,  1991,  p.  864). 

Table  I  Companies  Selling  3D  Audio  Products  (Wright,  1996,  p.  94) 


Analog  Devices 
Aureal  Semiconductor  Inc. 
Binaura  Corp. 
Creative  L  abs  Inc. 
Crystal  River  Engineering 
Crystal  Semiconductor  Corp 
DSP  Group  Inc. 

E-mu  Systems  Inc. 

ESS  Technology  Inc. 
Hannan  Interactive  Group 


Motorola 
NEC  Electronics 
OnChip  Systems  Inc. 
Panasonic  Industrial  Corp 
Qsound  L  abs  Inc. 

Seponix  Corp 

Spatializer  Audio  Laboratories  Inc. 
SRS  Labs  Inc. 

Yamaha  Systems  Technology  Inc. 
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There  are  individual  differences  in  HRTF  (Figure  4)  and  localization  performance  is  poorer  when 
using  someone  else’s  HRTF.  There  are  two  approaches  to  minimizing  this  error  1 )  mathematical  manipulation 
such  as  averaging,  structural  modeling,  or  principal  components  analysis  or  2)  use  the  HRTF  of  a  “good 
localizer”  (P  gault,  1991). 


sheets  i 


Figure  4.  HRTF  spectra  for  two  different  persons: 
left  ear,  source  at  0°  azimuth,  0°  elevation  (Begault,  1991,  p.864). 


Uses  of  3D  Audio 

3D  audio  has  been  used  to:  1 )  improve  intelligibility,  2)  provide  navigation  cues,  3)  warn  of  threats,  4) 
support  targeting,  5)  indicate  location  of  wingman,  and  6)  fly  an  aircraft.  Evaluation  of  each  of  these  uses  is 
described  in  the  follow  ing  sections. 

Improve  Intelligibility 

Begault  (1993)  reported  a  6  to  7  dB  intelligibility  improvement  using  3D  audio  over  monaural 
listening.  The  call  signs  were  presented  at  60  and  90  degree  light  and  right  positions.  The  task  was  identifying 
the  call  signs  of  1 30  aircraft.  This  finding  is  not  unique  NASA  (Begault,  1 995)  has  consistently  shown  about  a 
6  dB  improvement  in  intelligibility  through  the  use  of  3D  audio  communications. 

The  US  Army  (Haas,  Gainer,  Wightman,  Couch,  and  Shilling,  1997)  compared  the  number  of  correct 
pilot  responses  (i.e.,  pilot  replied  on  the  target  radio  channel  when  a  target  message  was  present)  in  three  radio 
signal  presentation  modes:  diotic,  dichotic,  and  3D  audio.  In  the  diotic  mode,  speech  messages  from  three 
simulated  radios  were  routed  to  both  ears  equally;  in  the  dichotic  mode,  speech  messages  from  two  simulated 
radios  were  routed  to  one  ear  and  the  third  radio  to  the  other  ear,  and  in  the  3D  mode,  the  three  radios  were 
presented  one  each  at  90°,  270°,  and  315  °  azimuth.  Data  were  collected  in  the  Army  Research  Institute 
Simulation  Training  Research  Advanced  Testbed  for  Aviation  (STRATA)  simulator.  The  subjects  were  1 1  US 
Army  helicopter  pilots  certified  in  the  AH-64  helicopter.  The  subjects  performed  the  radio  identification  task 
while  performing  target  acquisition  and  responding  to  aircraft  malfunctions.  The  results  showed  significantly 
better  performance  using  the  3D  audio  (5.0)  than  diotic  displays  (2.0)  currently  used  in  helicopters. 
Performance  for  dichotic  displays  (3.9)  was  between  the  other  two  displays. 

Ericson,  McKinley,  Kibbe,  and  Francis  (1993)  reported  a  25%  enhancement  in  intelligibility  of 
spatially  separating  competing  speech  messages  in  high  ambient  noise  environments  (115  dB).  Also  in  flight, 
separating  the  source  of  multiple  radio  signals  enhanced  intelligibility  in  an  AV-8B  (  Jane’s  Information  Group, 
1991). 

Provide  Spatial  Orientation  and/or  Navigation  Cues 

Pilot  critiques  indicate  that  a  3D  audio  signal  of  taxiway  location  may  enhance  terminal  area 
productivity  by  decreasing  time  to  taxi  from  runway  to  gate. 

Threat  Warning 

The  US  Marines  flight  tested  3D  audio  displays  in  an  AV-8B  in  the  Fall  of  1991 .  The  displays  were 
those  developed  by  the  US  Air  Force's  Armstrong  Laboratory.  The  test  evaluated  the  utility  of  these  displays 
for  warning  of  missile  approach.  Results  indicated  that  missiles  could  be  located  within  10  degrees  (Jane’s 
Information  Group,  1991). 
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US  Air  Force  researchers  (McKinley,  et  at.,  1995)  reported  that  subjects  could  detect  a  monochrome 
silhouette  of  an  SU-27  aircraft  with  the  naked  eye  as  well  as  with  a  Helmet  Mounted  Display  if  3D  audio  cueing 
was  used.  Rated  workload  (NASA  Task  Load  Index)  was  lowest  in  the  3D  audio  condition  as  compared  to  no 
sound  oi  non-localized  sound  conditions. 

NASA  has  shown  a  500  ms  improvement  in  acquiring  targets  using  a  3D  audio  version  of  the  Traffic 
Alert  and  Collision  Avoidance  System 
Targeting 

Pilots  participating  in  the  3D  audio  AV-8B  display  flight  tests  reported  targeting  accuracy  within  15 
degrees  azimuth  which  they  felt  was  adequate  to  orient  toward  a  target  (Kibbe  and  Francis,  1994).  However, 
elevation  cues  w  ere  less  accurate  and  enabled  only  rough  judgments  of  low  or  high.  Fricson,  McKinley  ,  Kibbe, 
and  Francis  (1993)  reported  that  in-flight,  3D  audio  reduced  target  acquisition  times. 

The  US  Air  Force  (Perrott,  Cisneros,  McKinley,  and  D’Angelo,  1995)  in  a  series  of  laboratory 
experiments  reported  significantly  shorter  search  times  for  targets  using  3D  audio  cues.  The  worst  performance 
occurred  at  +/-  150  degrees  azimuth  but  even  that  performance  was  better  with  than  without  3D  audio.  Perrot, 
et  al.  (1991)  had  reported  a  similar  enhancement  with  3D  audio  in  a  two-alternative  visual  search  task  (Figure 
5). 
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Figure  5.  Target  search  latency  with  and  without  3D  audio  cueing. 
(Perrott,  et  al.,  1995,  pp,  105  and  106) 


NASA  (Begault  and  Pittman,  1995,  1996)  compared  the  acquisition  time  of  targets  using  the  standard 
head-down  Traffic  Alert  and  Collision  Avoidance  System  (TCAS)  and  a  3D  audio  presentation  of  the  same 
information.  The  subjects  were  ten  two-person  crews  composed  of  airline  pilots  rated  in  Boeing  757,  767,  737- 
300400,  or  747-400  aircraft.  Data  were  collected  in  the  NASA-Ames  Crew-Vehicle  Systems  Research  Facility 
Advanced  Concepts  Flight  Simulator.  The  results  indicate  a  500  ms  improvement  in  acquiring  targets  using  a 
3D  audio  version  of  the  TCAS  (2. 1 3  s)  rather  than  the  standard  TCAS  (2.63  s). 

The  500  ms  improvement  has  also  been  reported  in  simple  laboratory  search  tasks  (Strybel,  Boucher, 
Fujawa,  and  Volp,  1995).  This  improvement  occurred  24  degrees  from  the  fixation  point  of  five  subjects  in  an 
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audiometric  chamber  for  a  70  dB  audio  cue.  The  improvement  was  slightly  less  (300  ms)  for  a  40  dB  audio 
cue. 

Endsley,  Rosiles,  Zhang,  and  Macedo  (1996)  examined  the  effects  of  tone  type  (pure,  oscillating, 
variable),  tone  number  (single,  with  reference  tone,  graduated),  frequency  type  (high,  low),  and  subject  type 
(fighter,  transport  pilot).  The  dependent  variables  were  time  and  accuracy  of  identifying  the  location  of  an 
auditory  signal.  The  smallest  elevation  error  occurred  with  the  variable,  graduated,  high  frequency  tone  There 
was  no  effect  of  type  of  subject.  Accuracy  was  poorest  at  elevations  below  zero  or  at  azimuths  above  70 
degrees.  The  smallest  azimuth  error  occurred  with  the  oscillating,  reference,  high  frequency  tone.  In  addition, 
fighter  pilots  had  smaller  azimuth  errors  than  transport  pilots.  Response  times  were  longest  when  a  reference 
tone  was  present  and  in  high  frequencies.  There  were  no  significant  effects  for  elevation  but  for  azimuth, 
response  time  decreased  from  the  subject’s  far  left  to  directly  in  front  of  him.  Ironically,  subjects  performed 
better  with  the  high  frequency  tone  but  preferred  the  low  frequency  tone. 

Wightman  (1995)  reported  that  auditory  cues  did  not  enhance  the  localization  of  moving  targets. 
However,  giving  the  subjects  control  of  the  target  movement  decreased  the  number  of  front/back  reversals.  He 
also  reported  that  there  were  no  differences  in  localization  performance  between  using  HRTF  measurements 
made  with  open-canal  probe  microphones  or  closed -canal  insert  microphones. 

Julig  and  Kaiwai  (1995)  are  evaluating  the  use  of  3D  auditory  displays  to  enhance  sonar  operator 
performance. 

Locate  Wingman 

A  method  has  been  developed  of  indicating  location  of  wingman  to  a  lead  pilot  using  outputs  from  the 
aircraft's  GPS  receivers  to  establish  their  relative  location. 

Fly  Aircraft 

Forbes  (1946)  modified  an  auditory  display  called  the  Flybar.  The  Flybar  presented  a  single  tone  that 
changed  in  intensity  between  the  pilot’s  two  ears  to  indicate  turn  rate  and  bank  angle.  The  presentation  rate  of 
the  tone  indicated  the  airspeed.  Similar  displays  have  been  developed  more  recently,  for  example  the  Acoustic 
Orientation  Instrument  (AOl)  developed  by  Lyons,  Gillingham,  Teas,  Ercoline,  and  Oakley  (1990).  Endsley, 
Rosiles,  Zhang,  and  Macedo  (1996)  tested  an  auditory  Head  Up  Display  (HUD)  in  which  pitch  was  presented  as 
the  vertical  location  of  the  tone  and  bank  as  the  horizontal  location  of  the  tone.  The  subjects  were  instructed  to 
maintain  their  flight  path  and  to  search  for  visual  targets.  Graduate  variable  broadband  tone  or  the  single  tone 
was  associated  with  smaller  RMS  altitude  error.  There  were  also  significant  subject  differences.  But  there  were 
not  any  significant  differences  in  auditory  tone  or  subject  for  Root  Mean  Square  (RMS)  heading  error.  The 
same  results  occurred  for  the  visual  search  task.  However,  there  were  again  significant  subject  differences. 
Neither  task  was  associated  with  significant  differences  in  the  Situational  Awareness  Global  Assessment 
Technique  (SAGAT)  scores.  The  Auditory  HUD  was  then  tested  in  an  unusual  attitude  recovery  task.  There 
was  no  significant  effect  of  tone  on  response  time  or  response  accuracy. 

Implementation  Problems  with  3D  Audio 

Problems  of  implementation  include: 

1 .  Dual-channel  equalization  -  for  the  human  to  detect  direction,  it  is  critical  that  the  sound  in  each  ear 
is  equalized  prior  to  the  delivery  of  the  3D  signal;  this  requires  cross-talk  cancellation  in  the  ear  phones; 

2.  Vibration  -  reduces  hearing  perception  especially  at  high  vibrations  (100,000  Hz); 

3.  Noise  -  for  signal-to-noise  ratios  less  than  about  15  dB,  noise  can  make  localization  more  difficult, 
this  is  especially  true  of  pure  tones; 

4.  Communication  -  the  same  earphones  used  for  the  3D  signal  are  used  for  communication  and  there 
have  been  some  problems  of  acceptance  by  transport  pilots; 

5.  Postural  adaptation  -  after  head  rotation,  the  perception  of  center  is  displaced  in  the  direction  of  the 
original  rotation, 

6.  Cones  of  confusion  -  3D  audio  requires  temporal  disparity  between  signals  to  the  left  and  right  ear. 
Small  or  no  disparities  indicate  that  the  sound  is  emanating  from  the  vertical  plane  between  the  two  ears, 
anywhere  in  this  plane.  The  greatest  confusion  is  up/down  and  ffont/back.  Front/back  reversals  are  common, 
back/front  less  so.  For  example,  Begault  and  Wenzel  (1993)  reported  1 1%  back/front  reversals  compared  to 
47%  ffont/back.  The  task  was  an  auditory  target  localization  task  in  a  sound  isolation  chamber. 
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7.  Intracranial ly  heated  sound  -  “inability  to  hear  sound  outside  of  the  head’1  (Begault,  1992,  p.  895). 
This  can  be  minimized  using  spatial  reverberation,”  HRTF  processing  is  applied  not  only  to  the  direct  sound, 
but  to  the  indirect  sound  field  as  well  .  “The  purpose  for  doing  so  is  to  allow  modification  of  the  original 
recording  so  that  it  seems  to  have  been  recorded  within  an  arbitrary  environmental  context”  (Begault,  1992,  p 
896).  A  sample  of  reflection  intensities  is  given  in  Figure  6.  Spatial  reverberation  decreases  intracranially  heard 
sound  but  increases  the  magnitude  of  azimuth  and  elevation  localization  errors  (Figure  7). 


Figure  6.  Reflection  Intensities  Over  Time  (Begault,  1992,  p.897) 
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Figure  7.  Difference  in  absolute  errors  for  azimuth  and  elevation  (Begault,  1992,  p.  901 ) 

8.  Nonlinearities  in  amplification  -  these  can  change  the  time  and  intensity  differences  that  provide 
binaural  cues;  and 

9.  Inconsistent  donning  of  headphones  -  this  may  result  in  a  mismatch  with  the  HRTF. 

For  systems  using  loudspeakers  rather  than  earphones,  another  set  of  problems  arise.  The  first  is 
crosstalk,  i.e.,  interference  between  signals  from  the  two  loud  speakers.  Transaura!  stereophony  has  been  used 
to  cancel  or  eliminate  crosstalk  by  precisely  controlling  the  acoustic  signals  at  the  listeners’  ears.  Bauck  and 
Cooper  (1996)  identify  several  advantages  of  transaural  stereo  1)  accurate  images  i.e.,  the  primary  sources  of 
sound  “may  be  heard  anywhere  around  the  listener  if  they  were  present  during  the  performance”  (p.  683)  and  2) 
accurate  spaciousness,  i  e.,  “accurate  placement  of  secondary,  or  reflected,  sounds  is  tantamount  to  proper 
spatial  rendering”  (p.  683). 

To  simulate  binaural  images,  binaural  image  synthesis  can  be  done  from  multitrack  recordings 
combined  with  crosstalk  cancellation.  Shuffler  filters  are  the  most  economical  way  to  perform  binaural  image 
synthesis  (Bauck  and  Cooper,  1996). 

Test  Vehicle 

Problems  of  implementation  are  being  addressed  in  the  installation  of  3D  audio  in  the  Variable 
Stability  In  Flight  Simulator  (VISTA).  The  VISTA  (Figure  8)  is  the  newest  concept  in  the  development  and  use 
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of  in-flight  simulation  VISTA  has  independent  control  of  five  degrees  of  freedom  of  motion  by  directly 
commanding  surface  deflections  and  engine  thrust  through  the  F-16  fly-by-wire  control  systems  The  airframe 
front  cockpit  has  been  modified  to  be  the  evaluation  cockpit  with  a  variable-feel  centerstick  and  with  simulation 
controls  and  displays.  The  safety  pilot’s  cockpit  (rear)  has  been  modified  to  permit  control  and  alteration  of  the 
simulation  in  flight  and  to  also  be  the  primary  command  and  flight  control  station  for  the  basic  F-16  host 
airplane 


m 


Figure  8.  VISTA 
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INTRODUCTION 


Background 


In  recent  years  information  in  the  cockpit  has  proliferated  and  become  more  sophisticated.  The 
integration  of  weapon,  navigation  and  communication  systems  has  resulted  in  information  management 
issues  of  interpretation,  dissemination,  and  utilization.  It  is  nearly  impossible  for  the  aircrew  to  utilize  all 
of  the  information  provided  by  advanced  technology  capabilities  due  to  the  current  inability  to  effectively 
synthesize  and  present  information  to  the  aircrew.  This  lack  of  information  management  not  only 
impacts  safety  of  flight  and  aircrew  survivability  but  reduces  the  operational  envelope  of  the  aircraft 
These  factors  and  issues  must  be  addressed  to  properly  configure  advanced  cockpit  systems  for  enhanced 
mission  effectiveness  and  increased  aircraft/aircrew  survivability.  Future  tactical  aircraft  cockpits  must 
provide  the  aircrew  with  the  information  and  interfaces  necessary  to  maximize  aircraft  performance  and 
fully  utilize  advanced  weapons  and  the  myriad  of  information  available  from  on-board  and  off-board 
sensors. 

There  is  significant  research  and  development  currently  being  performed  on  various  types  of  cockpit 
display  hardware  (e.g.,  HUD’s,  MFD’s,  HMD’s),  however,  little  work  has  been  performed  in  the  area  of 
information  fusion  and  information  management.  Helmet  mounted  displays  in  future  aircraft  will  not  only  save 
weight  and  reduce  costs  but  can  also  potentially  increase  the  operational  capability  of  the  aircraft  by  improving 
the  information  management  capabilities  of  the  pilot.  To  date,  research  focused  on  determining  what 
information  should  be  presented  on  the  HMD,  when  that  information  should  be  presented,  and  formats  for 
information  presentation  has  been  limited  During  the  Effective  Information  Fusion  for  HMD  Technologies 
program,  empirical  data  has  been  collected  on  the  performance  benefits  of  various  types  of  information 
presented  in  an  HMD.  This  data  will  aid  in  an  objective  selection  of  effective  HMD  information  content  for 
operational  implementation. 


Purpose 


The  Effective  Information  Fusion  for  Helmet  Mounted  pisplay  Technologies  program  has  four  major 

objectives: 

( 1 )  Develop  and  demonstrate  integrated  HMD  information  concepts. 

(2)  Empirically  demonstrate  performance  benefits  of  HMD  information  display  concepts  using  pilot-in-the- 
loop  simulation. 

(3)  Identify  technology  gaps  and  avionics  requirements  that  need  addressed  to  achieve  proposed  HMD 
formats. 

(4)  Develop  an  HMD  image  base  for  effective  information  management.  To  this  end,  a  multi-media  CD-ROM 
will  be  used  to  present  dynamic  HMD  images  that  demonstrate  effective  information  fusion,  avionics 
requirements  to  produce  those  images,  identification  of  HMD/information  management  technology  gaps, 
and  quantitative  performance  benefits  of  the  HMD  images. 

Expected  payofls  are  directly  related  to  the  Navy’s  needs  for  technical  solutions  to  operational  requirements. 

The  specific  payoffs  listed  below  are  consistent  with  those  identified  in  the  Human  Systems  Interface  Defense 

Technology  Area  Plan'. 
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•  Higher  accuracy  ofT-boresight  targeting  resulting  in  a  10:1  improvement  in  exchange  ratio/lethality  and 
survivability.  Provide  real-time  re-targeting  capability  in-flight. 

•  Lower  latency  sighting/lock-on  capability  resulting  in  a  doubling  of  first  pass  target  acquisition  and  kill 
capability. 

•  Improved  accuracy  of  visually  coupled  systems  resulting  in  decreased  reliance  on  the  head-up  display  (with 
associated  cockpit  design  advantages),  improved  targeting  and  navigation  update  performance. 

•  Advanced  control/display  crewstation  integration  and  flexible  “open”  avionics  architecture  designs 
characterized  by  increased  throughput  and  reduced  latency. 

•  Improved  aircrew  safety  and  surv  ivability  through  a  control 'display  system  configuration  compatible  with 
advanced  crew  systems  life  support  and  escape  technologies.  Higher  “G”  tolerance  and  integrated  loss-of- 
consciousness  detection  capability,  coupled  with  an  optimized  visual  interface  which  contribute  to  expanded 
flight  performance  envelopes. 


Test  Equipment 

Helmet  Mounted  Display 

The  helmet-mounted  display  used  during  this  evaluation  was  a  Hughes  Training  CiearVue  system  (Figure  1). 
The  system  consisted  of  a  helmet  shell,  display  unit,  and  a  separate  electronics  rack.  The  CiearVue  display 
provided  full  color,  high-resolution  display  of  symbology  and  video  in  a  “see-through”  mode  which  allowed  the 
user  to  also  view  the  out-the-window  scene  of  the  simulation.  The  CiearVue  system  has  an  advertised 
resolution  of  1280x1024  pixels  for  each  eye  and  a  total  field-of-view  of  80  degrees  horizontal  by  60  degrees 
vertical. 


Flight  Simulator 

The  flight  simulations  and  empirical  evaluations  were  performed  in  an  advanced  technology  cockpit 
simulator  at  the  Boeing  Integrated  Technology  Demonstration  Laboratory.  This  facility  contained  a  30- 
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foot-diameter  dome  display  with  an  advanced  fighter  cockpit  mounted  in  its  center  point.  The  Evans  & 
Sutherland  ESIG  4500  projection  system  consisted  of  three  background  projectors  which  provided  a 
high-resolution  forward  inset  and  a  lower  resolution  full-dome  out-the-window  scene  for  air-to-air  use. 
A  single  8:  ^  color  cathode  ray  tube  (CRT)  monitor  flanked  by  vo  6x6  color  CRT  moniti  s  graphically 
presented  the  head-down,  dynamic,  interactive  PVI  display  formats.  The  entire  simulation  was  generated 
using  a  Silicon  Graphics  Onyx  2  with  16  processors  and  3  graphics  pipes.  A  viewing  and  simulation 
control  room  was  on  an  elevated,  enclosed  platform  behind  the  dome  cockpit.  Visitors  and  engineering 
personnel  viewed  the  cockpit  activity  in  this  area  while  monitoring  and  examining  the  entire  spectrum  of 
technical  and  mission  aspects  of  the  simulation  in  progress. 


SYMBOLOGY  DESCRIPTIONS 


Baseline  Symbology 

Virtual  HUD  -  The  Virtual  HUD  provided  a  baseline  HUD  configuration  which  approximated  a  MIL-STD 
1787B  configuration  for  basic  flight  symbology.  The  virtual  HUD  symbology  was  presented  directly  on  the 
helmet  mounted  displa>  in  the  aircraft-stabilized  position  normally  occupied  by  a  physical  HUD  combiner.  A 
depiction  of  the  Virtual  HUD  symbology  is  provided  in  Figure  2. 


Figure  2:  Virtual  HUD  Symbology 
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Additional  air-to-air  weapon  symbology,  common  to  the  F/A-18  aircraft,  was  also  displayed.  This  symbology 
consisted  of  a  Normalized  In-Range  Display/Allowable  Steering  Error  circle,  Steering  Dot,  and  Target 
Designator  Box  with  SHOOT  Cue2.  The  baseline  air-to-air  symbology  is  depicted  in  Figure  3. 


Figure  3 .  Baseline  A/A  Symbology' 


Advanced  Symbology 

Several  concepts  were  integrated  to  form  an  advanced  symbol  set  for  the  air-to-air,  air-to-ground  and  navigation 
master  modes.  These  concepts  were  designed  to  utilize  the  unique  characteristics  of  helmet  mounted  displays 
including  the  large  display  field  of  regard  and  egocentric  display  presentation. 

Air-to-Air  Symbology 


The  air-to-air  (A/A)  symbology  consisted  of  four  major  components  which  combined  to  present  an 
intuitive,  egocentric  representation  of  the  air  battle  picture.  These  components  included  the  3D  Target 
Icon,  Locator  line/Reflected  Cue,  Weapon  Field  of  Regard  Boundary,  and  Caged  Display. 

The  3D  Target  Icon  (depicted  in  Figure  4)  identified  the  line  of  sight  to  a  target  as  well  as  information  on 
target  range,  altitude,  aspect  and  relative  weapon  range. 


Figure  4:  3D  Target  Icon  Symbology* 

Note  Target  range  for  relative  range  bar  was  calculated  using  a 
generic  unclassified  missile  model. 
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The  center  3D  aircraft  icon  showed  target  attitude  by  movement  about  its  3  rotational  axes.  The  icon 
appeared  as  the  actual  target  would  have  if  it  were  visible.  Threat  or  friendly  aircraft  were  indicated  by 
icon  color  (red=threat,  blue=friendly)  and  by  shape 

The  miniature  Normalized  In-Range  Display  (N1RD)  circle  around  the  aircraft  icon  indicated  target  range 
relative  to  the  range  of  the  selected  missile.  The  heavy  portion  of  tl.v  cucle,  or  relative  range  ^ar,  rotated 
clockwise  around  the  circle  as  target  range  increased.  This  range  was  computed  relative  to  the  fixed 
locations  of  max  and  mm  missile  range.  The  bottom  caret  represented  max  missile  range  (Rmax),  the 
center  caret  was  no  escape  range  (Rne),  and  the  top  caret  was  minimum  missile  range  (Rmin).  The  color 
of  the  N1RD  circle  was  red  for  highest  priority  target  (next-to-shoot)  and  green  for  all  other  targets  in  the 
shoot  list. 

A  “SHOOT1  cue  and  alphanumerics  indicating  target  altitude  and  range  were  also  associated  with  the  3D 
Target  Icon  Symbology. 

The  entire  symbol  set  (target  icon,  N1RD  circle,  alphanumerics,  etc.)  was  visible  for  the  highest  priority 
target  anytime  that  target  was  within  the  HMD  field  of  view.  For  other  targets,  symbol  sets  appeared 
only  when  that  target  was  within  n  10  degree  radius  of  HMD  boresight  and  the  target  ID  hands  on  throttle 
and  stick  (HOTAS)  switch  was  depressed.  In  the  case  of  secondary  targets,  the  N1RD  circle  was 
displayed  in  green  to  distinguish  them  from  the  primary  target. 

The  locator  line  (Figure  5)  was  designed  to  give  an  observer  quick  and  accurate  line-of-sight  or  “look-to1' 
oriented  guidance  toward  a  selected  point  of  interest3.  In  the  air-to-air  targeting  application,  the  locator 
line  indicated  the  continuously  computed  azimuth  and  elevation  vector  to  the  highest  priority  target  in  the 
shoot  list.  The  main  purpose  of  the  locator  line  was  to  indicate  the  relative  position  of  a  target  being 
tracked  within  the  sensor  field-of-regard  (FOR)  when  the  actual  target  location  was  beyond  the  display 
fleld-of-view  (FOV).  The  line  was  anchored  at  the  center  of  the  display,  at  the  aimsight  reticle,  and 
radiated  outward  toward  the  edge  of  the  display  drawing  surface.  The  locator  line  and  the  associated 
designator  symbology  superimposed  over  a  target  did  not  coexist.  TTe  locator  line  was  replaced  by  the 
designator  symbology  whenever  the  real  world  target  location  was  within  the  display  FOV.  Likewise, 
the  designator  symbology  was  replaced  by  a  locator  line  as  it  crossed  outside  the  display  FOV. 
Additional  locator  line  features  indicated  the  continuously  computed  angular  distance  between  the  target 
and  the  aimsight  reticle  (observer  LOS)  as  well  as  relative  target  aspect,  range,  and  weapon  firing 
solution  status. 
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Changes  in  angular  distance  (degrees  of  arc)  between  the  target  and  the  aimsight  reticle  were  represented  by  a 
dynamic  reflected  cue  symbol  that  moved  along  the  locator  line  in  proportion  to  the  rate  and  magnitude  of 
angular  distance  change.  The  length  of  the  visible  drawn  line  represented  a  compression  of  the  maximum 
angular  distance  the  target  can  be  tracked  beyond  the  HMD  FOV.  The  position  of  the  reflected  cue  on  the 
drawn  line  represented  the  instantaneous  angular  distance  of  the  target  within  the  maximum  available  distance. 
The  compression  mechanized  into  the  symbology  results  in  a  natural  slowing  of  the  apparent  motion  of  the  cue 
along  the  line.  This  was  important  to  reduce  ballistic  head  movement  overshoots  due  to  the  high  rates  of  closure 
resulting  at  typical  head  movement  velocities.  The  cue  moved  out  toward  the  edge  of  the  display  FOV  as  the 
angular  distance  between  the  aimsight  reticle  and  the  target  location  was  decreased  In  this  way,  the  cue 
appeared  to  be  a  “reflection”  of  the  converging  target  location.  The  observer’s  visual  attention  was  intuitively 
led  with  the  cue  along  the  locator  line  to  the  point  where  the  designator  symbology  crossed  into  the  HMD  FOV. 

Because  the  observer’s  attention  would  be  most  likely  concentrated  on  the  reflected  cue  symbology, 
additional  features  were  included  to  add  targetAveapon  status  information  to  the  tracking  task.  The 
primary  component  of  the  reflected  cue  was  formed  of  a  filled  shape  and  color-coded  symbol  consistent 
with  the  coding  strategies  employed  in  the  head-down  tactical  situation  displays.  In  the  case  of  an 
airborne  target,  the  symbol  represented  the  aspect  of  the  target  relative  to  ownship  flight  path  A 
miniature  N1RD  circle  surrounded  the  reflected  cue.  The  symbol  and  line  color  were  consistent  with  the 
identification  convention  used  throughout  the  cockpit 

The  Weapon  FOR  Boundary  symbology  was  designed  to  map  the  weapon  launch  volume  in  an 
egocentric  manner  onto  the  outside  scene  to  afford  quick  comparison  of  airborne  target  position  to  the 
limitations  of  the  selected  weapon.  The  symbology  viewed  in  its  entirety  would  appear  as  a  large  double 
ring  drawn  in  space.  Viewed  through  the  more  limited  FOV  of  the  HMD,  the  symbology  appeared  as  a 
portion  of  an  arc  (Figure  5).  The  inner  circle  was  formed  of  a  segmented  line  to  indicate  “inside”  the 
launch  parameter  volume.  A  solid  line  outside  ring  indicated  the  launch  volume  boundary.  Using  this 
concept,  a  target  symbol  superimposed  over  its  position  in  the  real  world  can  be  naturally  related  to  the 
selected  weapon  launch  limitations.  For  this  evaluation  the  Weapon  FOR  Boundary  was  designed  to 
behave  as  an  Allowable  Steering  Error  circle,  dynamically  changing  in  size  according  to  changes  in 
weapon  limitations  due  to  ownship  maneuvering,  target  range  changes,  target  maneuvering,  etc.  The 
symbology  could  also  be  mechanized  to  represent  only  the  missile  seeker  field  of  regard.  This 
mechanization  would  be  useful  when  incorporated  with  future  high  off-boresight  short-range  missiles. 

The  A/A  Caged  Display  (Figure  5)  represented  priority  air-to-air  targets  within  the  HMD  FOV  using 
symbology  conventions  consistent  with  those  applied  within  the  head-down  displays.4  Triangle  shape 
codes  were  employed  to  indicate  air-to-air  target  location  and  relative  aspect  angle.  The  target 
symbology  was  superimposed  directly  over  the  real  world  target  location  whenever  the  location  was 
within  the  HMD  FOV.  Targets  being  tracked  within  the  sensor  FOR  but  beyond  the  display  FOV  were 
drawn  caged  along  the  edge  of  the  display  FOV  at  the  proper  azimuth  and  elevation  extrapolation.  To 
keep  caged  symbols  from  being  confused  with  superimposed  symbols,  a  small  conformal  line  was  drawn 
through  the  caged  symbol  to  represent  the  appropriate  portion  of  the  FOV  edge  against  which  the  symbol 
was  caged.  Additional  symbology  features  were  employed  to  convey  basic  identification  information 
and  changes  in  target  state.  For  example,  color-coding  was  used  to  indicate  team  membership.  A 
numeral  associated  with  an  air-to-air  symbol  indicated  the  target’s  order  in  the  shoot  list,  A  flashing 
target  accompanied  by  an  auditory  warning  indicated  that  target  was  located  within  ownship’s  rear 
quadrant,  providing  a  “check  six”  alert  Symbol  size  was  increased  for  any  target  sensed  to  be  in  a  track 
or  firing  mode. 

Air-to-Ground  Symbology 

The  air-to-ground  (A/G)  HMD  concepts  consisted  of  Locator  Lines  and  a  Caged  Display  similar  to  those 
used  in  the  air-to-air  mode  along  with  head  steered  targeting  and  navigation  Forward  Looking  Infrared 
(FLIR)  sensors.  The  Locator  Lines  and  Caged  display  differed  from  the  air-to-air  equivalents  only  in  the 
symbology  used  to  represent  the  targets. 
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The  Integrated  Targeting  FLIR  concept  (Figure  6)  integrated  targeting  pod  forward-looking  infra-red  (T- 
FLIR)  imagery  with  the  out-the- window  (OTW)  scene,  represented  by  either  natural  terrain  or,  in  a 
night/all  weather  condition,  by  head-steered  navigation  FLIR  (Nav-FLIR)  imagery. 


Figure  6:  Head-Steered  TFLIR  Presented  m  the  HMD 


The  pilot  was  presented  with  a  head-stabilized  aiming  reticle  and  T-FLIR  field-of-view  (FOV)  indicators 
in  the  center  of  the  HMD.  Once  the  pilot  visually  acquired  a  target,  he  would  center  it  within  the  aiming 
reticle  and  command  a  “magnification”  by  a  HOTAS  switch.  As  the  pilot  looked  at  the  target,  the  T- 
FLIR  system  correlated  head-position  and  moved  its  line-of-sight  to  the  proper  location,  using  its  narrow 
FOV  capability  to  provide  the  pilot  with  a  magnified  image  in  the  center  of  the  HMD,  A  second  HOTAS 
switch  activation  moved  the  T-FLIR  image  inset  window  to  an  off-set  position  in  the  HMD  and 
commanded  the  T-FLIR  sensor  to  a  ground  track  stabilized  mode.  This  allowed  the  pilot  to  resume 
normal  head  movement  while  the  target  FLIR  image  remained  available  in  the  HMD  for  further 
evaluation  or  target  designation  purposes.5 

Missile  Evasion  Symbology 


Missile  evasion  symbology  was  available  in  both  the  A/A  and  A/G  modes  to  help  the  pilot  plan  and 
execute  missile  defense  maneuvers.  This  symbology  consisted  of  a  locator  line,  which  directed  the 
pilot’s  line  of  sight  to  the  missile,  integrated  with  a  missile  type  symbol,  which  identified  the  inbound 
missile  as  either  IR  guided  or  radar  guided.  An  alphanumeric  that  displayed  the  missile  time-to-impact 
was  also  associated  with  the  missile  symbol. 

Navigation  Symbology 

In  addition  to  the  Virtual  HUD  symbology,  which  was  used  in  conjunction  with  all  the  other  symbology 
(except  pathway-in-the-sky),  three  other  concepts  were  evaluated  which  were  designed  to  aid  the  pilot  in 
performing  the  basic  flight  task. 

Synthetic  terrain  was  used  to  aid  the  pilot  during  a  low-level  terrain  following  mission  segment.  This 
concept  displayed  a  grid  pattern  which  overlaid  the  terrain  to  provide  an  artificial  representation  of  the 
earth’s  surface.  The  grid  pattern  contour  was  based  on  digital  terrain  elevation  data  and  displayed  50  feet 
above  the  actual  database  terrain.  The  synthetic  terrain  was  designed  to  reduce  the  pilot’s  dependence  on 
the  traditional  primary  flight  reference  instruments,  such  as  the  attitude  indicator,  altimeter,  and  airspeed 
indicator,  by  allowing  him  to  use  the  simulated  terrain  as  he  would  the  actual  visual  scene  for  attitude 
reference  and  spatial  awareness. 
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An  off-boresight  3D  Attitude  Reference  (Figure  7)  displayed  aircraft  attitude  to  the  pilot  while  he  was 
looking  to  either  side  of  the  aircraft.  It  was  designed  to  provide  attitude  maintenance  information  only, 
and  was  not  intended  as  a  reference  for  unusual  attitude  recovery.  The  display  consisted  of  a  3D  aircraft 
icon  which  repeated  ownship  attitude  (pitch  and  roll  only)  The  aircraft  icon  was  read  against  a  horizon 
ring,  which  was  displayed  around  the  animated  aircraft  and  referenced  to  actual  horizon  Airspeed  (left), 
altitude  (right)  and  heading  (top)  were  presented  as  standard  alphanumeric  indications.  The  3D 
icon/horizon  ring  symbology  changed  perspective  with  pilot  head  movement  such  that  the  attitude 
symbology  appeared  as  another  aircraft  “in  formation”  with  ownship.  This  was  an  “outside-in”  display, 
with  the  horizon  ring  always  remaining  congruent  with  the  actual  horizon. 


Figure  7:  Off-boresight  3D  Attitude  Indicator 


A  Navigation  FLIR  gave  the  pilot  a  wide  field-of-view,  head-steered,  FUR  image  for  pilotage  in 
conditions  of  reduced  visibility,  or  for  wide  area  target  search.  The  Nav-FLIR  image  was  presented  at 
one-to-one  magnification  and  geospatially  referenced  to  the  outside  scene.  Flight  and  targeting 
symbology  were  overlaid  on  the  Nav-FLIR  image  providing  the  pilot  with  visibility  and  situation 
awareness  nearly  equal  to  that  of  day-VFR  conditions. 

Approach  Symbology 

Two  concepts  for  carrier  approach  to  landing  guidance  were  compared  to  the  basic  glide  path  deviation 
needles  contained  in  the  Virtual  HUD. 


The  Pathway-in-the-Sky  concept  used  the  familiar  method  of  displaying  pathway  blocks  to  indicate  the 
intended  route  of  flight  (Figure  8).  Two  vertical  scales  near  the  top  of  the  display  indicated  angle  of 
attack  (left)  and  optimum  altitude  above  the  pathway  (right).  A  large  “return  to  path”  arrow  was 


Figure  8:  HMD  Pathv  ay’-ln-The-Sky’ 
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presented  in  the  top-center  of  the  display  if  ownship  strayed  more  that  five  hundred  feet  off  course. 

The  Approach  Path  Command  Guidance  concept  was  modeled  after  the  flight  director  guidance  cues 
found  on  many  civil  aircraft  (Figure  9). 


Figure  9:  HMD  Approach-Path  Command 
Guidance 

This  concept  utilized  a  red  chevron  symbol  to  display  commanded  roll  and  pitch  angle  The  chevron 
was  read  against  a  wedge  s>mbol  which  was  mechanized  as  an  aircraft  climb/dive  marker.  The  pilot  flew 
the  aircraft  to  match  the  top  of  the  wedge  sy  mbol  with  the  notch  of  the  chevron. 

Symbology  in  center  of  the  wedge  presented  angle  of  attack  (AOA)  information  using  a  circle  moved 
vertically  on  dotted  scale  -  up  to  indicate  higher  angles  of  attack  and  down  to  indicate  lower  AOA.  The 
aircraft  was  “on-speed”  when  the  circle  was  over  the  center  dot. 

METHOD 


Mission  Scenario 

An  air-interdiction  mission  scenario  was  developed  containing  segments  which  were  representative  of 
missions  that  current  and  future  tactical  aircraft  might  support.  The  major  mission  elements  included  an 
air-to-ground  Joint  Direct  Attack  Munition  (JDAM)  delivery,  a  reconnaissance  segment  (Recee)/low- 
level  target-of-opportunity  (TOO),  an  A/A  battle,  and  carrier  recovery. 


Participants 

Six  pilots  from  Naval  Air  Station  Lemoore  CA  participated  irt  the  study.  All  were  qualified  F/A-18  pilots  with 
three  having  additional  experience  in  A-6E  aircraft.  Total  flight  hours  of  the  participants  ranged  from  1300- 
3300  hours.  None  of  the  participants  had  any  significant  prior  experience  with  helmet  mounted  displays.  The 
subjects  participated  on  a  voluntary  basis  and  were  paid  travel  and  per  diem  expenses.  Two  participants  per 
week  traveled  to  the  Seattle  test  facility  and  each  was  in  the  simulator  for  five  to  six  hours  per  day  for  the  entire 
week. 


Training 

During  the  week-long  test  period  for  each  subject,  Mondays  were  fully  devoted  to  training.  Each 
participant  spent  approximately  five  hours  in  the  simulator  cab  learning  the  HMD  symbology  and 
mission  tasks.  During  the  Monday  training  period  the  participant  not  flying  ihe  simulator  was 
encouraged  to  watch  from  the  simulation  control  room  where  he  could  monitor  the  display  repeaters  and 
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discuss  the  HMD  display  concepts  with  project  personnel.  Each  participant  received  approximately  30-45  minutes  of 
refresher  training  before  beginning  the  day’s  evaluations  during  the  remainder  of  the  week.  Participants  were  not 
allowed  to  observe  each  other  during  actual  data  collection. 


Tei  i  Matrix 


Overall,  the  evaluation  was  divided  into  part  task 
evaluations  and  a  full  mission  evaluation,  Due  to  the 
limited  time  available  to  perform  this  evaluation,  a 
building-block  approach  was  chosen  to  compare  the 
part  task  conditions.  The  part  task  evaluations  began 
with  a  baseline  condition  which  was  representative  of 
current  tactical  aircraft  capability.  Additional  display 
concepts  were  added  to  the  baseline,  each  building  on 
the  previous,  until  all  the  selected  advanced  concepts 
were  incorporated.  The  full  mission  evaluations 
consisted  of  a  baseline  condition  and  an  advanced 
condition  which  contained  all  of  the  advanced  display 
concepts.  The  part  task  evaluations  were  performed  on 
Tuesday  -  Thursday  of  the  test  week  and  the  full 
mission  evaluation  was  performed  on  Friday. 

The  carrier  approach  display  concepts  were  evaluated  at 
the  end  of  the  full  mission  conditions.  Since  there  were 
three  carrier  approach  concepts  and  only  two  full 
mission  conditions,  an  additional  carrier  approach  was 
flown  after  every  other  full  mission  condition. 

The  order  of  the  part  task  mission  segment  presentations 
was  randomized  across  subjects.  The  order  of 
presentation  of  the  conditions  within  the  mission 
segments,  both  part  task  and  full  mission,  was  also 
randomized  The  conditions  for  the  part  task  and  full 
mission  evaluations  are  listed  in  Tables  1-4. 


Table  I :  Air-to-Groui:d  Part  Task  Test  Conditions 


Condition  Display  Concepts 

A/G  Baseline 
(HUD) 

Virtual  HUD  symbology 

I  leads -down  T-FLIR 

A/G  H  MD  1 

Virtual  HUD  symbology 

Head  steered  T-FLIR 

A/G  HMD  2 

Virtual  HUD  symbology 

Head  steered  T-FLIR 

A/G  I  ook-to  Line 

Table  2:  Recee/TOO  Part  Task  Test  Conditions 


Condition  Display  Concepts 

Recce  Baseline 
(HUD) 

.  .J  . 

Virtual  HUD  symbology  w/  HUD  FLIR  display 

T.F  Box 

Head-down  T-FLIR  video 

Recce  HMD  1 

'  V  • 

Virtual  HUD  symbology 

T  F.  Box 

I  lead  Steered  Nav  FI  IR  and  T-FI  IR 

Recee  HMD  2 

■ 

v 

Virtual  I IUD  Symbology 

Synthetic  Terrain 

Head  Steered  Nav  FI  IR  and  T-FLIR 

Recce  HMD  3 

A'r  V-'rS  V; 

Virtual  Hi  ID  Symbology 

TF  Box 

3D  Attitude  Indicator  (Nav  mode  only) 

Head  Steered  Nav  FLIR  and  T-FLIR 

Recce  HMD  4 

- 

Virtual  HUD  SymboIogy/3D  Attitude 

Head  Steered  Nav  FLIR  and  T-FI  IR 

Synthetic  Terrain 

Recce  HMD  5 

. 

■ 

Virtual  FU  JD  Symbology  /3D  Attitude 

Head  Steered  Nav  FI  IR  and  T-FLIR 

Synthetic  Terrain 

Caged  Ground  Threat  Symbols,  Pop-up  Labels  and 
Threat  Symbols 
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Table  3:  Air-to-Air  Part  Task  Test  Conditions 


Table  4*  Full  Mission  Test  Conditions 


Condition 

Display'  Concepts 

A/A  Baseline 

V  ‘*ial  IIUD  symbology  with  traditional  A/A 

(IIUD) 

symbology 

A/A  HMD  1 

Virtual  MUD  symbology 

Locator  Line 

Simple  Target  Icon/ AS  E/Shoot  Cue 

A/A  HMD  2 

Virtual  1 IUD  symbology 
l^ocator  Line 

3D  Target  Icon/ASE/Shoot  Cue 

A/A  HMD  3 

Virtual  HUD  symbology 

Locator  Line 

3D  Target  Icon/ASE/Shoot  Cue 

A/A  Caged  Symbols/Chcck  Six  display 

A/A  HMD  4 

Virtual  MUD  symbology 

Locator  Line 

3D  Target  Icon/ASE/Shoot  Cue 

A/A  Caged  Symbols/Chcck  Six  display 

Weapon  Field -of- Regard  display 

A/A  HMD  5 

Virtual  IIUD  symbology 
l  ocator  Line 

3D  Target  Icon/ASE/Shoot  Cue 

A/A  Caged  Symbols/Chcck  Six  display 

Weapon  Ficld-of- Regard  display 

Missile  evasion  symbology  (highest  priority 
localor  line) 

Condition  Display  Concepts 


Full  Mission  I 
Baseline 

Carrier 

Approach  1 

All  Modes 

Virtual  MUD  symbology 

A/G  Mode 

MUD  FUR  display 

T  F.  Box 

Mead-down  T-FLIR  video 

A/A  Mode 

ASC  &  Steering  Dot/TD  Box/Shoot  Cue 

Full  Mission  2 

All  Modes 

Advanced 

Virtual  MUD  symbology 

Nav/II £  Modes 

Carrier 

3D  Attitudc/Pathway-in-thc-Sky 

Approach  2 

A/G  Mode 

Mead  Steered  Nav  I  LIR/T-FL  IR 

Synthetic  Terrain 

Caged  Ground  Threat  Symbols.  Pop-up  Labels  and 
Threat  Symbols 

A/G  Look  to  Line 

A/A  Mode 

Locator  1  ine  /ASE/Shool  Cue 

3D  Target  Icon 

; 

A/A  Caged  Symbols/Chcck  Six  display 

Weapon  Field-of-Regard  display 

Missile  evasion  symbology  (highest  priority 

locator  line) 

Carrier 

Nav/ILS  Modes 

Approach  3 

Virtual  HUD  with  Approach  Path  Command 

^  .  ‘  •  ^ 

Guidance 
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Measures  of  Merit 


The  m.asures  of  merit  recor  ed  during  the  simulations  are  listed  in  Table  5.  Pu^ts  also  completed  the  NASA 
Task  Load  Index  (TLX)  questionnaire  after  every  run 


Table  5  -  Measures  of  Merit 


Mission  Phase 

Measures 

Rec«/TOO 

-.Y- :  • . 

", 

Altitude  Deviation  (WP5  lo  6  only) 

Course  Deviation  (\\  P5  to  6  only) 

Airspeed  Deviation  (\VP5  to  6  only) 

Acquisition  Time  for  Each  TOO 

Designation  Time  for  Each  TOO 

Overall  Time  for  AequisitioaDesi  gnat  ion 

Designation  Accuracv  tor  Each  TOO 

Number  of  TOO’s  Acquired 

SAM  Exposure  Time 

Number  of  SAM  Tracks 

Number  of  SAM  launches 

Number  of  SAM  Hits 

Number  of  Ground  Strikes 

Fuel  Consumed 

Pilot  Workload  -  NASA  TLX 

I 

I 

I 

fMiSp’ 

h&BBK 

Fuel  Consumed 

SCUD  Acquisition  Time 

SCUD  Designation  Time 

Overall  Time  for  Acquisition/Dcsignalion 

SCUD  Designation  Accuracy 

SCUD  Acquisition  Success  (yes/no) 

Stand-Off  Range  al  Designation 

Pilot  Workload  -  NASA  TLX 

Alr-to-Air 

Exchange  Ratio 

Number  of  Threat  Radar  Tracks 

Threat  Exposure  Time 

Length  of  lime  threats  in  6  o’clock  position 

Fuel  Consumed 

Pilot  Workload  -  NASA  TLX 

Full  Mission 

All  Part-Task  Measures  (above) 

Timc-On-W'ay  point 

Glidcslope  Deviation  (Landing  only) 

Total  Fuel  Remaining 

Pilot  Workload  -  NASA  TLX 

Data  Analysis 

The  analysis  was  conducted  by  assessing  pilot  performance  data,  subjective  workload  ratings,  and 
interview  responses.  Prior  to  conducting  the  formal  objective  data  analysis,  frequency  distributions  of  all 
the  measures  of  merit  were  reviewed  in  search  of  outliers.  Data  points  greater  than  two  standard 
deviations  away  from  the  distribution  mean  were  corrected  using  the  same  mean  value.  This  procedure 
was  conducted  only  once  and  no  outlier  patterns  were  detected. 

Upon  completion  of  the  descriptive  analysis,  statistical  comparisons  of  the  performance  levels  and  workload 
ratings,  using  MANOVA  techniques,  were  performed  across  the  different  PV1  Configuration  and  Replication 
conditions.  Because  the  Off-Boresight  3D  Attitude  Indicator  was  limited  to  Navigation  Mode  and  the  Target 
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Locator  Line/Pop-Up  Labeling  was  presented  after  target  designation,  data  associated  with  TOO  and  SCUD 
acquisition/  designation  were  collapsed  across  HMD  conditions  based  on  FLIR  mechanization  (head-down  vs. 
head-up). 

Post-hoc  Analysis  of  Variance  (ANOVA)  and  Tukey-HSD  tests  were  performed  on  the  significant  effects  to 
identify  the  nature  of  the  statistical  differences.  Subjective  data  collected  aurmg  the  post-mission  and  exit 
interviews  were  reviewed  and  summarized.  The  following  sections  provide  the  relevant  results  of  the 
evaluation 

RESULTS  AND  DISCUSSION 
General 

Results  will  be  discussed  by  mission  segment.  A  short  description  of  each  task  is  provided.  Data  from 
the  part  task  conditions  and  the  associated  full  mission  segments  will  be  covered  together  in  order  to 
more  easily  associate  data  with  a  specific  display  concept  Significant  or  trend  data  given  will  be  denoted 
as  either  from  the  part  task  or  full  mission  runs. 

Across  all  concepts,  the  use  of  color  in  the  HMD  was  considered  highly  desirable.  Color  facilitates  the 
quick  determination  of  friendly  from  foe  and  provides  an  intuitive  relationship  between  heads  up  and 
heads  down  display  symbology.  It  was  noted  by  most  pilots  however,  that  the  number  of  colors  used  in 
the  HMD  should  be  kept  to  a  minimum. 

Recee/TOO 

Task 


The  Recee/TOO  task  contained  a  low-level  (200  ft.),  terrain  following  flight  task  in  conditions  of  poor 
visibility.  During  this  segment  pilots  were  comparing  the  functionality  of  the  terrain  following  (TF)  box 
with  synthetic  terrain.  During  this  low-level  segment,  pilots  came  under  fire  from  several  surface-to-air 
(SAM)  missile  sights  located  along  the  route.  They  were  to  attempt  to  avoid  being  hit  by  the  SAMs, 
using  missile  evasion  symbology  in  the  HMD  in  one  condition  and  using  heads  down  missile  defense 
symbology  in  the  other  conditions. 

After  completing  the  low  level  run,  the  pilots  climbed  to  1000  feet  AGL  and  commenced  a  search  for 
pre-briefed  targets  of  opportunity.  Using  a  targeting  FLIR,  either  manually  steered  (via  the  throttle 
designator  control)  or  head  steered,  the  pilots  located,  identified  and  designated  various  targets.  During 
the  pre-brief  they  were  shown  screen  shot  “photos”  of  the  intended  targets  which  they  were  to  pick  out. 
Distracter  targets  were  also  present  in  the  visual  database. 

Results  and  Discussion 


Pilots  generally  preferred  the  synthetic  terrain  to  the  terrain  following  HUD  cue.  They  found  it  more 
intuitive  to  fly  although  nearly  all  of  them  noted  that  the  terrain  grid  tended  to  interfere  with  the  HUD 
symbology,  especially  at  lower  altitudes  where  the  terrain  lines  tend  to  be  very  close  together.  The  most 
prevalent  complaint  about  the  TF  box  was  that  it  did  not  provide  a  good  predictive  capability  of  the 
upcoming  terrain. 

Objectively,  the  synthetic  terrain  showed  one  key  advantage  over  the  TF  box.  SAM  exposure  time  and 
number  of  SAM  tracks  were  lower  during  the  full  mission  scenario  which  contained  the  synthetic  terrain 
(Figure  9).  This  indicates  that  pilots  were  able  to  use  the  predictive  nature  of  the  synthetic  terrain  to  alter 
the  route  of  flight  slightly  in  order  to  obtain  terrain  masking  from  the  SAM  threats. 
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Figure  9:  SAM  Exposure  Time 


Pilots  overwhelmingly  preferred  the  head  steered  T-FL1R  to  the  manually  steered  system  The  ability  to 
quickly  designate  targets  while  staying  heads  out  of  the  cockpit  allowed  them  to  maintain  greater 
situational  awareness.  The  only  negative  comments  associated  with  the  head  steered  T-FL1R  were 
simulator  related  The  resolution  of  the  T-FLTR  image  in  the  HMD  was  lower  than  on  the  heads  down  T- 
FLIR  display,  making  identification  of  the  pre-briefed  targets  more  difficult  in  the  HMD,  T  his  complaint 
was  validated  by  the  fact  that  the  target  designation  accuracy  was  much  worse  during  the  advanced  full 
mission  scenario  using  the  head  steered  FLIR. 

Objective  results  of  the  part-task  evaluation  indicated  that  pilots  deviated  significantly  less  from 
commanded  altitude  (1000  ft.  AGL)  with  the  advanced  condition  consisting  of  the  head-steered  T-FLIR 
presented  in  the  HMD  than  with  the  baseline  condition  consisting  of  the  head-down  T-FLIR  (Figure  10). 
This  would  indicate  that  providing  the  T-FLIR  image  in  the  HMD  allowed  the  pilots  to  cross-check  flight 
information  more  efficiently. 


Figure  10;  Altitude  Deviation  During  TOO  Acquisition 


Although  not  significant,  pilots  tended  to  take  less  time  to  designate  a  TOO  using  the  head-steered  T- 
FL1R  (advanced  condition)  than  with  the  head-down  T-FLIR  (baseline  condition)  (Figure  1 1).  The  head- 
steered  cueing  capability  provided  by  the  HMD  enabled  the  pilot  to  slew  the  T-FLIR  over  a  target  and 
designate  much  quicker  than  with  the  traditional  HOTAS  mechanization.  Further  evidence  of  this  is 
provided  by  the  significant  results  found  in  the  Air-to-Ground  task. 


Figure  11:  TOO  Designation  Time 
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Air-to-Ground 


Task 


The  A/G  part  task  evaluations  consisted  of  a  target  acquisition  task  during  which  the  pilot  searched  for  a 
SCUD  target.  Though  the  target  position  was  pre-briefed,  the  actual  target  position  in  the  database 
differed  from  the  pre-briefed  position  to  simulate  a  mobile  target.  After  locating  and  designating  the 
target  with  the  T-FLIR,  the  pilot  delivered  a  JDAM  weapon  on  the  target.  As  in  the  Recec/TOO 
conditions,  a  manually  steered  T-FLIR  was  used  for  the  baseline  configuration  and  a  head  steered  T- 
FL1R  for  the  advanced  conditions. 

Results  and  Discussion 


Pilot  opinion  concerning  the  head  steered  T-FLIR  was  even  more  positive  during  the  A/G  phase  of  the 
evaluation.  The  capability  to  look  at  a  target  and  accept  or  reject  it,  and  designate  when  appropriate,  was 
unanimously  well  received.  As  in  the  Recee/TOO  phase,  the  only  negative  comments  had  to  do  with  the 
poor  T-FLIR  display  resolution  in  the  HMD.  Target  designation  accuracies  were  again  better  using  the 
head  down  display. 

The  objective  data  for  the  A/G  evaluations  indicated  that  target  acquisition  times  and  target  designation 
times  were  significantly  better  with  the  head-steered  T-FLIR  (advanced)  than  with  the  head-down  T- 
FL1R  (baseline)  (Figure  12). 
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Figure  12:  A/G  Acquisition  and  Designation  Time 

Using  the  pilots  subjective  comments  as  a  guide  for  interpretation,  the  head-steered  cueing  capability 
provided  by  the  HMD  allowed  the  pilots  to  quickly  slew  the  T-FLIR  over  possible  targets  and  evaluate 
using  its  magnification  capability.  By  enabling  the  pilot  to  locate/evaluate  targets  more  rapidly, 
acquisition  time  and  thus  designation  time  for  the  actual  target  was  reduced. 

Overall  subjective  workload  ratings  (mental  demand  and  performance  estimation  specifically)  were 
significantly  lower  for  the  advanced  condition  with  head-steered  T-FLIR  and  target  locator  line  than  for 
the  baseline  condition  with  head-down  T-FLIR.  This  indicates  that  the  advanced  condition  reduced 
workload  by  allowing  the  pilot  to  more  easily  return  to  and  attack  the  target  once  it  was  designated. 

Air-to-Air 

Task 

The  A/A  task  consisted  of  a  3  friendly  vs.  5  hostile  engagement.  The  other  two  friendly  aircraft  w  ere 
controlled  by  the  simulation  and  always  engaged  the  same  two  hostile  aircraft.  The  subject  pilots  were 
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responsible  for  the  remaining  three  bogeys  and  were  briefed  on  which  aircraft  to  engage.  As  they  entered 
the  A/A  engagement  zone  they  saw  only  four  of  the  five  hostile  aircraft  -  the  fifth  was  a  pop-up  threat 
that  was  programmed  to  appear  at  the  subject  pilot’s  six-o’clock  position  after  a  certain  time  had  elapsed 
The  pop-up  fhreat  was  designed  to  evalu  *e  the  functionality  of  the  “check  six”  portion  of  the  A/A  caged 
display. 

Results  and  Discussion 


Pilot  comments  regarding  information  presented  by  the  advanced  concept  A/A  symbology  were  generally 
favorable.  All  subjects  seemed  to  like  the  locator  line  symbol  for  look-to  guidance  to  the  target  and  the 
use  of  the  NIRD  circle  as  the  target  designator  “box,”  All  but  one  also  favored  the  3D  target  icon  over  the 
basic  triangle  icon  as  the  target  designation  symbol,  and  that  subject  showed  signs  of  changing  his  mind 
as  the  week  progressed.  All  of  the  subject  pilots  noted  a  requirement  that  the  3D  target  icon  become 
transparent  or  even  be  removed  as  the  actual  target  comes  into  visual  range. 

Comments  on  the  caged  display  were  mostly  favorable  but  less  numerous.  Due  to  problems  in  accurately 
fitting  the  HMD  to  all  subjects,  the  periphery  of  the  display  was  sometimes  difficult  to  see  Since  the 
caged  display  was,  by  design,  always  near  the  display  edge,  it  was  often  difficult  to  see. 

During  the  full  mission  evaluations,  the  number  of  threat  radar  tracks  and  the  number  of  ownship  deaths 
were  both  significantly  less  in  the  advanced  condition  (Figure  13).  This  could  indicate  that  pilots  had 
greater  situation  awareness  when  using  the  advanced  A/A  HMD  concepts.  Pilots  often  noted  the  SA 
improvement  of  being  able  to  look  at  a  target  and  get  range  information  from  the  NIRD  circle  and  being 


#  of  Threat  Tracks  #  of  Ow  nship  Deaths 


Figure  13  Number  of  A/ A  Threat  Tracks  and  Ownship 
Deaths 

able  to  anticipate  target  movements  from  the  3D  target  icon.  The  mean  time  a  threat  was  in  ownship’s 
six  o’clock  position  was  also  significantly  less  when  the  advance  HMD  concepts  were  available.  This 
would  indicate  that  the  pilots  were  able  to  achieve  some  benefit  from  this  display  despite  it  being  difficult 
to  see. 
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Though  not  statistically  significant,  an  interesting  trend  was  shown  in  the  pilot  workload  ratings  for  the 
air-to-air  part  task  evaluations  (Figure  14). 


Mean 

Weighted 

Workload 

Scores 


Q  A/A  Baseline  ■  A/A  HMD  1  □  A/A  HMD  2 

□  A/A  HMD  3  ■  A/A  HMD  4  □  A/A  HMD  5 

Figure  14:  A/A  Workload  Scores 


Workload  showed  signs  of  decreasing  as  some  of  the  advanced  symbology  was  added,  but  as  more 
symbology  was  added  workload  began  to  increase  again.  This  trend  would  indicate  that  in  some 
conditions  the  symbology  became  cluttered  and  more  difficult  to  use. 

Carrier  Approach 

Task 

The  carrier  approach  evaluations  were  performed  in  conjunction  with  the  full  mission  simulations.  At  the 
end  of  the  baseline  full  mission  condition,  the  pilots  performed  a  carrier  landing  using  the  standard 
glideslope  deviation  needles.  At  the  end  of  the  advanced  full  mission  condition  they  performed  a  carrier 
approach  using  the  pathway-in-the-sky.  A  third  approach  symbology  condition  was  given  after  every 
other  full  mission  condition. 

The  carrier  approach  phase  seems  to  be  the  one  mission  aspect  upon  which  pilot’s  felt  there  was  really  no 
improvement  to  be  gained  with  any  of  the  advanced  symbology  evaluated.  There  were  some  positive 
comments  regarding  the  ease  of  flying  the  pathway  once  established  on  it.  Several  of  the  pilots  felt  that 
the  mechanization  of  the  pathway  in  this  evaluation  could  have  been  improved  upon  In  this  evaluation, 
pilots  obtained  all  attitude  information  from  the  orientation  of  the  pathway,  there  was  no  back-up  attitude 
symbology  available.  If  the  pilot  strayed  too  far  from  the  pathway,  there  was  no  attitude  information 
available.  Most  pilots  also  commented  that  the  pathway  would  have  been  easier  to  fly  if  the  altitude  and 
angle-of-attack  scales  had  been  further  down  toward  the  middle  of  the  display  near  the  end  of  the  path 
blocks,  rather  than  near  the  top  of  the  display.  There  were  no  differences  in  airspeed  or  glideslope 
deviation  between  the  standard  approach  symbology  and  the  pathway. 

There  were  no  positive  comments  whatsoever  regarding  the  approach  path  command  guidance.  It  was  by 
far  the  most  maligned  and  disliked  concept  presented  during  this  evaluation.  The  primary  complaint  was 
that  it  was  too  large  and  blocked  the  view  of  the  carrier.  Another  complaint  was  that  it  only  provided 
command  guidance  to  the  glidepath  and  did  not  provide  information  on  the  magnitude  of  the  glidepath 
error.  Glideslope  deviation  was  not  significantly  worse  with  this  concept  than  the  other  two  approach 
symbologies,  however  airspeed  deviation  was. 

SUMMARY 

Helmet  mounted  displays  will  undoubtedly  become  a  major  element  of  the  of  the  cockpit  display  suite  in 
future  tactical  aircraft  In  fact,  basic  HMD  systems  are  already  being  incorporated  into  many  aircraft. 
The  need  for  extensive  research  in  helmet  display  formats  is  ongoing. 
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The  primary  goals  for  this  project  were  to  evaluate  the  basic  requirements  for  helmet  mounted  display 
information  content  and  identify  the  technological  shortfalls  which  must  be  addressed  to  meet  the 
information  requirements.  To  this  end,  the  Effective  Information  Fusion  for  Helmet  Mounted  Displays 
Technologies  program  has  demonstrated  that  the  HMD  can  be  used  to  provide  an  intuitive,  egocentric 
display  medium  which  can  greatly  increase  the  situation  awareness  of  the  tactical  aircraft  pilot. 

The  authors  believe  there  are  two  issues  of  paramount  importance  when  considering  the  design  of  future 
helmet  display  formats.  First,  developers  of  display  formats  must  not  lose  sight  of  the  unique  nature  of 
the  helmet  display  and  its  capability  to  present  information  in  an  egocentric  format.  The  HMD  should 
always  remain  first  and  foremost  a  weapons  cueing  and  sensor  display  device.  The  display  of  basic 
primary  flight  information  can  be  accomplished  as  well  by  traditional  or  modified  aircraft  referenced 
HUD  formats  (whether  displayed  on  a  traditional  HUD  combiner  or  in  an  HMD).  Big  picture,  ‘‘God’s 
eye”  situation  awareness  formats  arc  better  presented  on  heads  down  display. 

Second,  the  temptation  to  put  too  much  information  in  the  HMD  must  be  resisted  to  an  even  greater 
extent  than  in  HUD  symbology  design.  A  pilot  can  look  around  or  away  from  the  HUD  if  it  becomes  too 
cluttered  A  helmet  display  is  extremely  compelling  and  will  always  be  directly  in  the  pilot’s  line  of 
sight.  Too  much  information  or  symbology  in  the  HMD  could  distract  the  pilot  from  his  primary  task,  or 
even  w'orse,  obscure  important  information  or  visual  cues  available  in  the  real  world. 

Though  not  discussed  in  this  paper,  the  technological  requirements  which  must  be  addressed  in  order  to 
meet  the  information  display  requirements  set  forth  during  this  evaluation  will  be  covered  in  the  CD- 
ROM  desktop  demonstration.  The  CD-ROM  will  also  contain  interactive  multimedia  demonstrations  of 
the  concepts  evaluated  during  this  project  as  well  as  animations  of  many  concepts  which  we  were  unable 
to  incorporate  into  the  flight  simulation  due  to  time  and  budgetary  constraints. 
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Summary 

The  objective  of  the  present  research  is  to  enhance  pilot  performance  and  reduce  spatial 
disorientation  which  was  identified  as  the  major  cause  of  aircrew/aircraft  losses.  Spatial  disorientation 
tends  to  occur  when  a  pilot's  gaze  is  temporarily  directed  away  from  the  aircraft  feedback  instruments. 

Our  approach  is  to  use  a  haptic  interface  to  provide  continuous  and  non-visual  feedback  of  the 
aircraft  attitude  and  orientation  to  the  pilot.  This  can  be  done  by  using  an  elbow  force  feedback  system 
being  developed  at  Rutgers  University  for  human  biomechanical  rehabilitation 

The  sensation  of  attitude  is  provided  by  applying  a  torque  on  the  elbow  joint  to  resist  the 
flexion/extension  movement.  The  sensation  of  orientation  is  provided  by  applying  a  torque  to  resist  the 
rotation  of  the  forearm.  The  intensity  of  the  torque  is  proportional  to  the  deviation  from  the  optimal 
attitude  and  orientation.  Zero  torque  signals  to  the  pilot  that  the  attitude  and  orientation  are  correct  while 
high  torque  signals  that  the  aircraft  is  in  a  dangerous  attitude  or  orientation.  It  is  clear  that  high  torque 
should  not  exceed  25%  of  the  maximum  human  capability  to  move  the  elbow  in  order  to  let  the  pilot 
override  if  needed. 

The  elbow  force  feedback  unit  is  made  of  pneumatic  structural  fabric  located  around  the  elbow 
joint.  The  system  is  adapted  to  the  morphology  of  the  human  elbow  and  doesn't  reduce  its  motion  range. 
The  pneumatic  actuator  designed  for  rehabilitation  is  light  and  capable  of  producing  a  torque  up  to  35Nm 
at  30psi  which  is  enough  to  resist  the  forearm  flexion  and  rotation  of  most  men 

This  newly  designed  haptic  system  can  easily  be  adapted  to  the  anti-G  suit  and  aircraft  cockpit 
conditions.  Initial  experimental  tests  were  done  to  evaluate  the  improvement  of  the  situation  awareness 
when  information  of  attitude  and  orientation  of  the  aircraft  are  provided  hapticly  to  the  pilot. 


l  Introduction 

The  aircraft  cockpit  is  designed  so  that  all  the  instrument  information  is  displayed  to  the  pilot 
visually.  In  a  normal  situation,  the  amount  of  significant  instrument  data  are  small  and  the  pilot  has 
enough  time  to  perceive  and  interpret  everything  before  making  a  decision  and  taking  action. 
Unfortunately,  in  situation  awareness,  the  significant  sensor  information  increases  and  the  pilot  has  to 
frequently  switch  attention  between  different  visual  displays,  which  is  demanding  and  can  cause  a 
dangerous  delay  in  reaction 

Also,  in  certain  situations,  the  pilot  can  be  temporarily  disoriented  and  his  gaze  can  be  directed 
away  from  the  altitude  and  the  orientation  display.  Without  this  critical  information,  the  risk  of  crash  is 
higher.  This  temporary  disorientation  was  identified  as  the  major  cause  of  aircrew/aircraft  losses  [  1  ]  [2]. 

In  these  cases,  the  presence  of  a  second  source  of  information  concerning  aircraft  attitude  can  be  very 
helpful  for  pilots.  One  of  these  sensorial  channels  uses  the  haptic  perception 

Our  interest  in  developing  haptic  interfaces  for  pilots  was  motivated  by  two  factors  Firstly,  the 
haptic  perception  of  the  pilot  is  underutilized,  while  the  visual  perception  is  overloaded  Secondly, 
humans  perceive  haptic  information  at  much  higher  bandwidth  than  visual  information,  and  the  pilot  can 
make  use  of  it  particularly  in  situation  awareness. 

There  is  not  much  research  being  done  in  this  area.  One  project  is  the  tactile  suit  developed  at 
the  Spatial  Orientation  Systems  Department  (NAMRL)  [3].  The  system  uses  vibrotactile  stimulators  to 
apply  attitude  and  motion  information  taken  from  the  aircraft’s  instruments  to  the  pilot’s  body  torso 
(Figure  1).  This  torso  suit  system  provides  continuous  non-visual  information  to  the  pilot. 
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Figure  I :  The  vibrotactile  body  torso  suit  [3] 

Haptic  feedback  was  also  provided  by  handling  the  aircraft  control  in  a  fighter  simulator  [4]. 
The  force  feedback  simulates  the  centrifugal  forces  when  the  aircraft  is  rolling  and  by  wind  forces  during 
turbulent  weather.  Experimental  results  showed  increased  landing  accuracy  compared  to  simulation  using 
a  control  stick  with  no  force  feedback  [5]. 

Most  of  the  haptic  feedback  interfaces  were  developed  for  virtual  reality  applications  like  tele¬ 
surgery  and  tele-rehabilitation  applications  [6].  The  research  of  our  laboratory  focuses  on  developing  a 
force  feedback  interface  for  hand  tele-rehabilitaion  and  recently  for  elbow  and  knee  tele-rehabilitation 
[7].  The  force  feedback  interface  proposed  in  this  paper  is  based  on  our  research  on  the  elbow  tele¬ 
rehabilitation  system.  Our  goal  is  to  use  the  elbow  interface  unit  to  increase  the  perceptual  understanding 
of  the  spatial  orientation  of  the  aircraft.  The  paper  presents  in  sections  2  and  3  the  concept  and  design  of 
the  elbow  interface.  Section  4  describes  the  structure  of  the  pneumatic  actuator  used  to  build  the 
interface.  Initial  experimental  results  evaluating  the  interface  for  spatial  orientation  are  presented  in 
section  5.  Conclusions  are  given  is  section  6. 

2  Spatial  orientation  using  an  elbow  haptic  interface 

The  concept  of  the  elbow  interface  is  based  on  the  application  of  torque  to  the  elbow  joint  in 
order  to  inform  the  pilot  about  the  deviation  of  the  aircraft  from  its  optimal  orientation  and  attitude.  The 
applied  torque  intensity  is  proportional  to  the  amplitude  of  the  deviation.  The  elbow  interface  can  apply 
two  torques  at  elbow  joint  axes  (Figure  2). 

The  First  torque  opposes  the  flexion  and  the  extension  of  the  forearm  which  is  interpreted  by  the 
pilot  as  a  deviation  of  aircraft  pitch  orientation  from  horizontal  reference  (or  other  optimal  reference). 
When  the  interface  torque  is  directed  to  open  his  elbow  joint,  the  pilot  should  react  naturally  to  this  force 
by  extending  his  forearm  until  the  torque  becomes  zero.  In  this  case,  the  pilot  arm  motion  pushes  the 
aircraft  stick  in  the  forward  direction,  which  corresponds  to  a  pitch  down  orientation  and  to  a  reduction  of 
the  aircraft  altitude.  In  the  other  case,  when  the  torque  tends  to  close  his  elbow  joint,  the  pilot  reaction 
should  be  natural  and  rotate  his  elbow  joint  in  order  to  reduce  the  torque.  This  elbow  rotation  corresponds 
to  a  backward  stick  motion,  which  increases  the  aircraft  altitude. 

The  second  interface  torque  is  perpendicular  to  the  flexion-extension  torque  and  is  directed  in 
order  to  turn  the  pilot  forearm  to  the  left  or  to  the  right.  When  the  torque  bends  the  pilot  forearm  to  the 
left,  the  pilot  should  react  in  the  same  direction  and  rotate  his  forearm  to  the  left  until  the  torque  becomes 
zero.  This  forearm  motion  corresponds  to  a  left  turn  of  the  aircraft  stick  or  a  left  roll  of  the  aircraft  with 
value  equal  to  his  deviation  from  optimal  orientation.  In  a  similar  way,  when  the  interface  actuator 
produces  a  torque  to  bend  the  pilot’s  arm  to  the  right,  the  pilot  should  then  rotate  the  stick  to  the  right 
which  corresponds  to  a  rotation  of  the  aircraft  to  the  right. 

Note  that  this  left/right  motion  doesn’t  correspond  to  a  natural  degree  of  freedom  at  the  human 
elbow  joint,  but  it  corresponds  to  the  aircraft  stick  motion  for  varying  the  aircraft  roll  orientation  In  the 
next  section,  we  will  describe  the  interface  design  and  we  will  see  that  the  user  feels  very  comfortable 
with  this  torque. 
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Figure  2:  Correspondence  between  the  aircraft  attitude  and  the  elbow  torque:  the  elbow  flexion/exten¬ 
sion  torque  corresponds  to  the  aircraft  nose  up/down  (pitch  angle)  and  the  elbow  left/right  torque 
corresponds  to  the  aircraft  banking  (roll  angle). 

3  Design  of  the  elbow  force  feedback  interface 

Physical  interface  for  pilots  should  assure:  1)  comfort  and  safety,  and  2)  unconstrained  natural  motion. 
The  first  design  goal  is  accomplished  by  developing  an  interface  without  using  mechanical  joints  nor 
mechanical  links.  The  unique  components  are  small  pressurized  bladders  which  make  the  interface  easy  to  be 
integrated  as  part  of  the  anti-G  suit.  Note  that  the  anti-G  suit  already  contains  a  pressure  system  to  prevent 
pooling  of  blood  during  high  accelerations.  The  pressure  system  is  built  with  several  air  bladders  and  can  apply 
positive  pressure  to  lower  body  extremities  of  pilots  [8]. 

The  second  design  goal  is  accomplished  by  developing  an  interface  made  of  two  separate  parts 
fixed  at  the  forearm  and  arm  links.  Thus,  the  area  near  the  center  of  elbow  joint  is  always  free  to  allow  a 
complete  and  natural  motion  of  the  pilot’s  arm.  With  such  design,  the  interface  simulates  the  elbow 
torque  instead  of  applying  a  real  torque  as  was  previously  designed  for  the  tele-rehabilitation  application. 
Therefore  the  pilot  always  has  control  of  his  arm  motion  and  aircraft  stick  even  in  the  presence  the 
feedback  torque.  Thus  the  interface  informs  the  pilot  about  the  aircraft  attitude  but  does  not  control  the 
aircraft  by  constraining  his  arm  motion. 

The  elbow  haptic  interface  is  made  with  eight  pneumatic  bladders  grouped  in  two  sets  of  four,  an  arm 
set  and  a  forearm  set  (Figure  3).  The  bladders  of  each  set  are  fixed  around  the  pilot  arm  link  at  90  intervals. 
Each  bladder  of  the  arm  set  is  coupled  to  a  bladder  of  the  forearm  set  located  on  the  opposite  side  at  1 80  When 
the  bladder  is  pressurized,  it  expands  in  balloon-like  fashion  against  the  pilot's  arm  and  maintains  a  directed 
force.  Therefore,  exerting  a  variable  pressure  (force)  among  the  bottom  bladder  of  the  arm  set  and  the  top 
bladder  of  the  forearm  set  (Figure  3-a),  creates  a  pair  of  forces  which  mechanically  correspond  to  the  elbow 
extension  torque.  On  the  other  hand,  pressurizing  the  top  bladder  of  the  arm  and  the  bottom  bladder  of  the 
forearm  (Figure  3-b)  creates  a  pair  of  forces  which  mechanically  correspond  to  the  elbow  flexion  torque.  In  a 
similar  way  the  left/right  direction  of  the  elbow  torque  is  created  with  the  left  and  right  bladders  (Fii»ure  3-c  and 
3-d). 
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Figure  3:  Distribution  of  the  pneumatic  bladders  near  the  elbow  joint 
a)  extension  torque,  b)  flexion  torque,  c)  left  torque,  d)  right  torque 

4  Structure  of  the  pneumatic  actuator 

The  interface  bladder  consists  of  an  expandable  internal  tube  surrounded  by  a  fabric  shell.  The 
inner  elastic  tube  is  made  of  silicon  rubber  material,  w  hich  has  the  capability  to  sustain  repeated  strains  of 
over  300%.  The  fabric  shell  acts  to  constrain  the  expansion  in  order  to  prevent  the  tube  from  blowing  out 
under  high  pressure. 

Fach  bladder  is  attached  to  its  own  thm  and  rigid  base  made  of  carbon  composite  material  (Figure  4-a 
and  5-b).  The  base’s  rigidity  keeps  the  force  direction  toward  the  pilot’s  skin  when  the  bladder  inflates.  Note 
that  the  base  has  a  small  bent  shape  in  order  to  avoid  a  permanent  contact  of  the  bladder  w  ith  the  pilot’s  skin 
when  no  pressure  is  present. 

The  four  bases  and  the  bladders  are  attached  together  with  four  thin  velcro  straps  to  form  a  cylindrical 
shape.  The  velcro  attachments  allow  the  adjustment  of  the  90°  interval  position  of  each  bladder  considering  the 
different  sizes  of  the  pilot’s  arm.  Also,  the  ring  shape  of  the  velcro  attachment  allows  the  pilot  to  feel  each 
bladder  separately  and  locate  the  direction  of  the  bladder’s  forces.  Figure  4-b  shows  the  force  distribution 
around  the  arm  when  only  one  bladder  is  pressurized. 

The  maximum  pressure  that  can  be  sustained  by  the  bladders  is  50psi.  To  increase  comfort,  we  limit 
this  pressure  to  30psi.  A  high-pressure  range  increases  the  sensitivity  of  the  interface  to  small  variations  of  the 
aircraft  attitude.  The  pneumatic  bladders  are  driven  by  proportional  pressure  regulators  that  operate  in  a  closed 
loop.  The  bandwidth  of  the  output  pressure  was  experimentally  measured  at  5  Hz.  These  units  are  controlled 
with  an  embedded  486  PC  which  can  communicate  with  any  host  computer  through  a  RS232  line  (Figure  5-a). 
This  control  unit  is  part  of  the  interface  normally  controlling  the  Rutgers  Master  hand  interface  [7], 

The  pressure  regulator  bandwidth  of  the  above  controller  may  be  insufficient  for  perception  of  spatial 
orientation  through  the  elbow  interface  at  the  maximum  pitch  and  roll  speed  motion  (10  rpm  speed  of  the  spatial 
disorientation  trainer).  A  new  pressure  regulator  which  increases  this  bandwidth  is  currently  being  developed  in 
our  laboratory.  This  new  regulator  uses  fast,  high  airflow,  solenoid  valves  controlled  by  software.  The 
bandwidth  of  the  output  pressure  is  estimated  at  20Hz  when  the  maximum  controlled  pressure  is  30psi  and  the 
input  pressure  is  lOOpsi. 
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Figure  4:  Illustration  of  the  pneumatic  actuator: 
a)  set  of  4  bladders,  b)  forces  distribution. 


Figure  5:  Elbow  interface:  a)  controller  unit,  b)  user’s  arm  wearing  the  interface  unit. 


5  Initial  experimental  study 

The  experiment  reported  here,  studies  the  effect  of  force  feedback  at  the  elbow  joint  on  controlling  the 
aircraft  attitude  when  visual  information  is  either  poor  or  absent.  To  achieve  this,  we  built  a  dedicated 
experimental  flight  system.  It  consists  of  a  graphics  station  (Pentium  300  with  Fire-GL  graphics  accelerator),  a 
2D  joystick  (a  replica  of  the  F15E  flight  stick)  and  the  elbow  interface  described  above.  The  joystick  and  the 
elbow  interface  communicate  with  the  graphics  station  through  a  serial  line,  and  receive  new  commands  at  60 
Hz. 

The  graphics  station  displays  an  animated  virtual  aircraft  attitude  indicator  implemented  using  Sense8 
WTK  software  library  [9]  at  60  fps.  The  virtual  attitude  indicator  displays  a  reference  line,  a  horizon  line,  and  a 
virtual  line  (Figure  6).  The  horizon  line  represents  the  aircraft  attitude  (aircraft  nose  down/up  and  banking 
right/left)  controlled  by  the  user  through  the  stick.  The  virtual  line  represents  the  target  or  the  desired  aircraft 
attitude  generated  randomly  by  computer. 
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Figure  6:  Illustration  of  the  Virtual  Attitude  indicator  displayed  on  the  PC 


The  maximum  dev  iation  of  the  target  line  (virtual  line)  from  the  reference  line  is  set  to  90°  (from  0°  to 
-90°  or  from  0°  to  90°  according  to  the  direction)  for  the  roll  and  pitch  orientation.  The  rate  of  change  in 
orientation  is  set  at  10°/sec,  which  is  close  to  the  real  attitude  rate  of  change. 

When  the  deviation  of  the  horizon  line  (controlled  by  user)  from  the  target  line  (controlled  by 
computer)  is  less  than  30°,  the  bladder  pressure  of  the  elbow  interface  is  proportional  to  the  attitude  deviation. 
Thus  an  attitude  deviation  of  20°  roll  left  will  correspond  to  pressure  of  20psi  on  the  corresponding  bladders. 
When  the  attitude  deviation  is  greater  than  30°,  the  corresponding  bladder  pressure  is  constant  and  equal  to 
30psi.  With  this  approach,  we  increase  the  resolution  at  small  attitude  deviations  and  expect  that  an  attitude 
deviation  greater  than  30°  will  be  an  infrequent  event.  When  the  target  line  is  aligned  with  the  virtual  line,  the 
bladder  pressure  is  equal  to  zero. 

During  the  experiment,  the  user  is  seated  in  the  front  of  the  PC  screen,  while  wearing  the  elbow 
interface  and  grasping  the  joystick  (Figure  7).  The  user  is  then  instructed  to  track  the  virtual  line  with 
visual  feedback  and  haptic  feedback.  The  user  performs  a  few  minutes  of  trials  to  become  familiar  with 
the  interface  (recognition  of  the  bladder  pressure  and  torque  distribution).  The  experimental 
measurements  were  recorded  from  several  trials  done  by  one  user. 


Figure  7:  Overview  of  the  experimental  site 


The  First  experiment  compares  the  performance  of  the  haptic  display  with  the  visual  display  in 
presenting  attitude  information  to  the  pilot.  The  experiment  consists  of  tracking  a  random  attitude 
trajectory  with  visual  feedback  alone  and  then  with  haptic  feedback  alone.  Figures  8-a  and  8-b  plot  the 
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target  and  user's  trajectory  for  pitch  deviation.  The  average  tracking  error  was  about  4  2°  for  the  visual 
feedback  and  6.4°  for  the  haptic  feedback.  The  average  tracking  error  with  only  haptic  feedback  is  close 
to  the  error  measured  when  using  visual  feedback,  which  means  that  the  information  sent  by  the  interface 
is  correctly  understood  by  the  user. 


Figure  8:  Pitch  orientation  tracking: 
a)  with  visual  feedback  only,  b)  with  haptic  feedback  only 


The  second  experiment  simulates  the  situation  awareness  when  pilot  gaze  is  temporarily  directed  away 
from  the  visual  display.  The  experiment  consists  of  tracking  a  random  attitude  trajectory  with  an  altered  visual 
feedback,  with  and  without  haptic  feedback.  The  altered  visual  feedback  is  simulated  by  switching  oa'off  the 
display  of  the  attitude  indicator  at  different  times.  The  switch-off  times  change  continually  from  0  (attitude 
indicator  is  permanently  displayed)  to  5  sec  after  60  sec  of  tracking,  while  the  switch-on  time  is  constant  and 
equal  to  0.2sec.  For  example,  at  30  sec  into  the  experiment  the  attitude  indicator  is  switched  off  for  2.5sec.  and 
switched  on  for  0.2sec. 

Figures  9-a  and  9-b  show  an  example  of  the  tracking  results  for  one  user.  The  average  tracking  error 
decreases  from  27.4°  without  haptic  feedback  to  8.9°  when  haptic  feedback  supplements  the  poor  visual 
feedback.  The  improvement  is  significant  when  the  haptic  feedback  is  present. 


Figure  9:  Roll  orientation  tracking: 

a)  altered  visual  feedback,  b)  altered  visual  supplemented  by  haptic  feedback 
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6  Conclusions 


A  simple  and  ergonomic  elbow  force  feedback  interface  designed  to  improve  the  spatial  orientation  of 
the  pilot  was  developed.  The  interface  has  the  advantage  that  it  allows  the  full  and  natural  motion  of  the  arm  and 
can  be  easily  integrated  with  the  pressurized  anti-G  suit.  The  interface  is  made  with  four  pairs  ol  pneumatic 
bladders  surrounding  the  arm  and  the  forearm  near  the  elbow  area.  When  the  bladders  are  pressurized,  a 
simulated  torque  is  created  at  the  elbow  joint  which  informs  the  pilot  about  the  aircraft  attitude  deviation  from 
an  optimal  or  a  reference  attitude. 

A  simple  evaluation  experiment  using  a  graphics  station  as  an  attitude  indicator  display  and  a  2D 
joystick  as  an  aircraft  stick  was  performed.  The  experiment  compared  the  performance  of  tracking  a  virtual 
attitude  randomly  generated  by  computer  using  visual  display,  with  and  without  haptic  feedback.  The  results 
show  a  real  improvement  in  the  understanding  of  the  attitude  information  when  the  haptic  feedback  was  present. 

More  experiments  should  be  performed  in  future,  using  a  more  sophisticated  flight  simulator,  to 
evaluate  precisely  the  improvement  in  reducing  the  pilot’s  spatial  disorientation  in  situation  awareness.  Further 
experiments  should  measure  the  benefit  of  using  a  faster  pneumatic  controller  currently  under  development  at 
Rutgers  University 
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Abstract:  The  Tactile  Situation  Awareness  System  (TSAS)  consists  of  an  array  of  tactors  applied,  in  columns 
and  rows,  to  the  torso,  that  provide  intuitive  three  dimensional  (3D)  information  to  aircrew,  astronauts  and 
operators  of  remote  operated  vehicles.  Intuitive  delivery  of  orientation,  velocity  and  range  has  been  proven  in 
actual  flight  tests  with  TSAS.  Due  to  the  integrative  nature  of  central  nervous  system  design,  haptic  interfaces 
enhance  and  supplement  audio  and  visual  displays.  Tactile  displays  have  already  demonstrated  synergistic 
effects  with  standard  flight  instruments,  improving  precision  in  areas  such  as  hovering  and  transitioning  to  and 
from  forward  flight  both  with  and  without  degraded  external  visual  conditions.  TSAS,  like  3D  sound,  provides 
360  degree  coverage  in  an  easy-to- interpret  format.  Furthermore,  it  can  resolve  some  of  the  illusions  and 
ambiguities  that  occur  when  using  3D  sound  displays  in  dynamic  environments  Applications  to  enhance  head 
up  displays  (HUD)  and  helmet  mounted  displays  (HMD)  include  drawing  attention  to  and  targeting  of  objects 
outside  the  field  of  view  tactilely  and  haptic  presentation  of  spatial  orientation  flight  parameters.  Such  a  system 
would  increase  situation  awareness  (SA)  while  allowing  reduced  clutter  in  a  visual  display.  Utilization  of  the 
presently  unused  haptic  sensory  channel  permits  reduced  visual  distraction  and  improved  resolution  of  audio 
cues.  A  multi-modal  system  promises  reduced  spatial  disorientation  and  pilot  workload. 

A  number  of  tests  in  fixed  and  motion  based  simulators  have  demonstrated  decreased  reaction  time, 
increased  precision  of  maneuvers  and  decreased  perceived  workload  when  tactile  cues  are  provided  in  addition 
to  the  normal  visual,  audio  and  motion  (on  motion  platforms)  information.  Flight  tests  in  fixed  and  rotary'  wing 
aircraft  have  proven  that  haptic  displays  can  supplement  cockpit  displays  and  external  cues  to  improve  SA  in 
flight.  Furthermore,  in  test  situations,  TSAS  has  supplanted  pilot  visual  systems,  when  visual  information  was 
excluded  In  flight,  relying  on  tactile  information,  pilots  have  reported  decreased  workload,  increased  SA,  and 
improved  maneuver  accuracy. 

Simulations  of  helicopter,  fixed  wing  aircraft,  astronaut  extravehicular  activity,  space  shuttle  landing, 
parachute  navigation,  high  speed  boat  and  undersea  diving  scenarios  have  been  developed.  Deployed  systems 
have  been  tested  in  a  Navy  T-34  turboprop,  an  Army  UH-60  and  by  special  forces  divers,  with  positive  results. 
TSAS  development  continues,  as  more  platforms  and  applications  are  identified  and  systems  are  deployed  into 
real  world  situations. 

Keywords.  Situation  Awareness,  Spatial  Disorientation,  Tactile  Situation  Awareness  System,  Extra  Vehicular 
Activity,  Three  Dimensional  Audio,  Workload. 

Introduction:  In  the  course  of  human  evolution,  the  sense  of  touch  has  developed  to  be  preeminent  in 
providing  immediate  and  intuitive  awareness  of  the  location  of  things  that  are  so  near  that  they  come  in  contact 
with  the  body.  The  adaptive  advantage  of  this  development  is  self-evident.  Tactile  interfaces  that  exploit  the 
unambiguous  and  vivid  awareness  of  location  on  the  body’s  surface  have  been  shown  to  improve  the  SA  of 
pilots  of  fixed  and  rotary  wing  aircraft  and  of  divers  in  undersea  environments.  The  tactile  situation  awareness 
system  (TSAS)  consists  of  an  array  of  tactile  transducers  (tactors),  held  in  contact  with  the  torso  of  a  human 
operator,  that  delivers  spatial  orientation  intuitively.  With  the  inherent  and  intuitive  body-referenced 
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organization  of  the  somatic  tactile  sense,  the  brain  readily  assimilates  position  and  attitude  relative  to  other 
significant  objects  in  the  environment.  Providing  orientation  information  in  this  manner  reduces  perceived 
workload  in  real  world  operations.  This  reduction  results  from  the  continuous  nature  of  the  TSAS  provided  data, 
which  is  maintained  even  wh'le  the  operator  focuses  on  a  specific  instrument  or  charges  radio  frequencies.  By 
exploiting  this  relatively  “unused”  sensory  pathway,  TSAS  allows  an  operator  to  concentrate  on  those  mission 
tasks  requiring  the  higher  resolution  senses  of  sight  and  sound  more  effectively. 

In  the  aviation  context,  tactile  interfaces  for  military'  fixed-wing  and  rotary-wing  aircraft  have  been 
developed  to  improve  the  spatial  orientation  and  SA  of  pilots.  These  tactile  interfaces  have  been  shown  effective 
and  intuitive  (i.e.,  minimal  training  required)  in  three  flight  test  programs.  An  underwater  diving  TSAS 
application  has  been  field  tested,  as  well  as  fixed  base  simulations  of  extravehicular  activity  (EVA),  space 
shuttle  landing,  parachute  navigation,  high  speed  watercraft  TSAS  applications.  While  TSAS  was  developed 
primarily  in  response  to  the  number  of  military  aircraft  mishaps  related  to  loss  of  SA,  work  continues  on  tactile 
interfaces  for  special  forces  missions  such  as  undersea  mine  countermeasures,  parachuting,  and  overland 
navigation.  The  sense  of  touch  has  proven  to  be  a  flexible  conduit  for  a  human-machine  interface  for  SA, 

Background:  The  first  and  most  reliable  inflight  SA  display  used  by  pilots  is  the  visual  world  outside  the 
cockpit.  When  these  outside  cues  are  ambiguous  (e  g.,  fog  or  brown  out/white  out  conditions),  the  pilot 
transitions  to  visual  flight  instruments  which  accurately  display  the  aircraft  spatial  orientation  (SO).  TSAS  was 
developed  in  response  to  an  alarming  number  of  SA  related  military  aviation  mishaps.  Even  conservative 
estimates  of  loss  of  materiel  and  human  resources  are  alarming  [Collins,  1995].  It  was  noted  that  spatial 
disorientation  (SD)  resulting  from  loss  of  SA  often  occurs  in  concert  with  an  element  of  visual  distraction 
[Murdock,  1998].  Since  present  SA  displays  in  military  aircraft  are  visual,  pilots  can  not  receive  critical  SA 
information  when  not  looking  at  the  display.  Existing  audio  warnings,  such  as  ground  proximity,  may  give 
coarse  SA  information  but  do  not  relay  an  accurate  portrait  of  the  aircraft  SO.  Proposed  solutions  have  included 
wide  field  visual  displays  (such  as  the  Malcolm  horizon)  or  three-dimensional  (3D)  audio  [Doll,  et.  a/., 1986]. 
While  much  effort  has  been  directed  toward  these  venues,  no  system  has  yet  been  successful  in  operational 
deployment.  Both  systems  have  demonstrated  deficiencies  in  the  evolving  operational  envelope  of  the  modem 
air  warfighter  Elements  such  as  night  vision  goggles  (NVG’s)  limit  field  of  view,  constraining  presentation  of 
any  SA  information  to  the  same  narrow-  visual  angle  (Figure  1).  One  problem  of  3D  audio  results  from  what 
Doll,  et  al  ( 1 986)  refer  to  as  a  zone  of  confusion  that  exists  for  a  subject  responding  to  synthetic  3D  audio  cues. 
Estimations  of  elevation  and  azimuth  in  this  conical  region,  extending  out  in  front  of  a  subject,  show 


Figure  1 :  Flead  up  display  symbology  superimposed  on  night  vision  goggles  (ANV1S). 

significant  error  across  a  wide  range  of  frequencies.  For  most  aviation  situations,  this  area  is  likely  to  be  of 
greatest  interest  for  maintaining  SA.  Tactile  cueing  could  be  integrated  with  visual  and  audio  situation  displays 
in  a  complementary  fashion,  possibly  improving  SA  synergist ically.  Previous  tactile  displays  that  have 
attempted  to  present  images  to  the  skin  [Bach-y-Rita,  1970,  White,  1970]  failed  due  to  the  relatively  low 
bandwidth  of  the  human  torso’s  touch  sense.  Others  have  tried  to  map  instrument  displays  on  the  torso 
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[Sanneman,  1975]  without  much  success.  The  TSAS  approach  has  been  to  exploit  three  dimensional 
distribution  of  the  tactile  sensory  system  for  three  dimensional  orientation  information. 

Because  the  sense  of  touch  conveys  location  on  the  body  of  a  tactile  stimulus  intuitively  [White,  1970],  an 
operator  utilizing  an  appropriately  designed  tactile  interface  imme  atc'y  knows  the  location  of  the  tactile 
transducer  (tactor)  as  it  fires.  As  such,  SA  is  maintained  at  a  low  cognitive  cost  as  compared  to  interpreting  a 
two  dimensional  visual  display.  Tactors  may  be  placed  anywhere  on  the  body  of  the  pilot,  and  objects  and 
motion  in  the  surrounding  three-dimensional  space  intuitively  maps  to  specific  tactor  locations.  Figure  2  shows 
a  conceptual  mapping  of  3D  space  to  tactor  locations  on  the  torso.  With  such  a  system,  pilots  can  rapidly 
interpret  three-dimensional  information  such  as  orientation,  obstacle  position,  object  location,  track,  etc.  While 
TSAS  strives  primarily  to  improve  SA,  additional  benefits  include  decreased  perceived  pilot  workload  and 
increased  accuracy,  evidenced  during  the  flight  test  evaluated  maneuvers 


Figure  2:  Three  dimensional  space  mapped  to  the  body. 

Methods:  The  TSAS  program  has  completed  three  flights  tests  (and  one  underwater  field  test)  for  proof  of 
concept.  In  the  first  test,  conducted  in  1995,  a  Navy  test  pilot-flight  surgeon  flew  a  series  of  maneuvers  in  a 
Navy  T-34C  Mentor  (fixed  wing,  two  seat,  tandem,  single  engine  turboprop  trainer). 

These  maneuvers  included  straight  and  level  flight,  standard  rate  turns,  unusual  attitude  (UA)  recoveries  and 
ground  controlled  approaches  (GCA).  In  addition,  two  aerobatic  maneuvers  were  performed,  aileron  rolls  and 
inside  loops.  Throughout  all  these  maneuvers  the  test  pilot  was  controlling  the  aircraft  from  the  rear  cockpit.  All 
instruments  were  removed  from  this  cockpit  (Figure  3)  and  the  instrument  flight  rules  (1FR)  training  hood 
prevented  use  of  external  visual  cues. 


Figure  3:  Rear  cockpit  of  T-34C  with  instruments  removed. 
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The  TSAS  apparatus  provided  aircraft  pitch  and  roll  angles  to  tactors  on  the  test  pilot’s  torso.  As  the  pitch  angle 
increased,  the  tactor  stimulus  frequency  would  increase  in  three  discrete  steps.  Each  step  indicated  transition 
from  one  preset  angle  range  to  the  next  (e.g.,  2.0°  to  3°,  then  3.0°  to  5°).  When  the  angle  exceeded  that  defined 
for  the  highest  frequency  on  a  given  tactor  location,  the  next  tactor  in  sequence  fired  instead  at  its  lowest 
frequency.  Each  of  16  tactors  were  encoded  this  way  for  both  pitch  and  roll.  With  this  system,  the  pilot  could 
determine  the  direction  to  the  earth’s  surface.  When  the  aircraft  was  in  straight  and  level  flight,  the  null 
condition,  no  tactors  were  active.  The  pilot  performed  all  the  required  maneuvers  with  the  addition  of  auditory' 
cues  for  altitude,  airspeed  and  g-force,  as  needed,  from  the  safety  pilot. 

In  the  second  flight  test,  a  UH-60A  (twin  turbine,  14  passenger,  multirole  helicopter)  provided 
the  rotary  wing  TSAS  proof  of  concept  for  non-hovering  flight.  The  same  set  of  maneuvers  were 
performed  by  U.S.  Army  research  pilots,  less  the  aerobatics.  The  pilots  wore  a  blackout  helmet  visor  to 
exclude  internal  and  external  visual  information  (Figure  4). 


Figure  4:  Blackout  helmet  visor  used  in  UH-60  flight  test. 


Pilots  in  this  test  program  received  pitch,  roll,  airspeed  error  and  heading  error  tactually  (a  total  of  20  tactors 
used),  with  additional  required  information  (such  as  altitude)  provided  via  audio.  Again,  each  tactor  encoded 
three  frequencies  corresponding  to  set  intervals  of  deviation  from  straight  and  level  flight  (for  airspeed/heading 
error,  deviation  from  desired  airspeed/heading  was  presented  in  three  interval  steps).  Two  research  pilots 
performed  the  described  maneuvers  in  the  blind. 

A  third  flight  test,  utilizing  the  same  UH-60  platform,  consisted  of  a  TSAS  proof  of  concept  for 
hovering  with  transitions  to  and  from  forward  flight  for  the  Joint  Strike  Fighter  program.  Four  research 
pilots  hovered  in  and  out  of  ground  effect,  translated  fore-aft  and  side  to  side,  and  transitioned 
successfully  to  and  from  forward  flight  in  simulated  shipboard  maneuvers.  Tactile  information  delivered 
to  the  pilots  consisted  of  drift  velocity  (direction  and  speed)  in  the  horizontal  plane.  Tactors  were  arrayed 
about  the  torso  in  a  ring  of  eight  (45°  apart).  The  position  Of  the  tactor  indicated  the  direction  of  the 
aircraft  motion  (nose  of  the  aircraft  relative  to  the  earth’s  surface).  A  differential  global  positioning 
system  (GPS)  provided  groundspeed  information  which  TSAS  delivered  in  three  steps,  0.2  to  1,  1  0  to  2 
and  2  0  or  greater  meters/second.  The  pilots  integrated  TSAS  data  with  either  full  internal  and  external 
visual  cues  as  in  normal  flight  or  with  full  internal  visuals  (instruments)  and  degraded  external  visual 
cues  (using  semi-fogged  spectacles,  ‘Toggles,”  equivalent  to  20/200  vision).  In  addition,  audio  cues  were 
provided  by  the  safety  pilot  as  needed. 

An  underwater  diving  field  test  consisted  of  a  simulated  mine  countermeasure  triangular  search  pattern. 
Special  forces  divers  utilized  a  two  tactor  system  (one  on  each  side  of  the  torso)  that  provided  course  deviation 
and  end  of  search  pattern  leg  information.  They  were  permitted  normal  vision  (murky  water  with  low  visibility  ) 
and  utilized  a  digital  readout  of  compass  heading  and  X-Y  position  relative  to  the  starting  point  of  the  search. 

In  addition  to  the  functional  tests,  numerous  ground  based  simulations  have  been  developed.  In  support  of 
the  Joint  Strike  Fighter  hover  flight  test,  a  UH-60  motion  model  running  on  a  desktop  simulator  (Silicon 
Graphics)  provides  a  visual  scene  along  with  driving  a  multifunction  display  patterned  after  that  from  the  MH- 
60K  helicopter.  Subjects  may  pilot  the  simulation  using  these  visual  displays  or  integrate  them  with  a  TSAS 
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display.  Similar  simulations  have  been  used  to  develop  TSAS  displays  for  underwater  diver,  surface  craft, 
parachutist  and  astronaut. 

The  astronaut  EVA  simulation  consisted  of  a  simplified  space  station  model  The  subject  was 
tasked,  as  a  suitably  equipped  astronaut,  with  three  dimensional  navigation  toward  a  target  astronaut 
(Figure  5). 


The  target  astronaut  was  placed  in  one  of  four  locations  and  was  either  on  a  straight  line  path  from  the  subject, 
on  a  straight  line  path  but  not  in  view,  on  an  indirect  path  with  part  of  the  space  station  preventing  a  straight 
path,  or  on  an  indirect  path  with  only  a  small  portion  of  the  space  station  in  view.  These  four  tasks  are  listed  in 
increasing  difficult,  but  were  presented  in  a  balanced  manner  to  12  subjects  to  control  for  order  effects.  Since  a 
limited  set  of  tactors  was  available  for  this  experiment,  only  6  tactors  were  used.  As  such,  one  to  three  were 
fired  in  order  to  represent  3D  space  (e.g.,  one  tactor  “front,’1  three  tactors  “front,  up,  left”)  The  control  task 
without  TSAS  consisted  of  the  same  sets  of  maneuvers,  but  with  voice  commands  directing  the  subject  toward 
the  target  (as  presently  practiced  b>  NASA  [Rochlis,  1998]). 

Discussion  In  the  first  two  flight  tests,  the  blindfolded  pilots  safely  completed  each  maneuver  with  greater 
accuracy  than  with  full  instruments  and  external  visuals  [Rupert,  et.  al ',  1996].  The  pilots  often  reported  SD 
symptoms  such  as  “the  leans”  while  in  the  blind,  but  could  easily  and  reliably  determine  the  SO  of  the  aircraft 
via  the  tactile  cues.  In  addition,  the  TSAS  equipped  pilot  did  not  lose  SA  when  performing  unusual  attitude 
recoveries,  unless  the  system  was  deactivated  during  set-up  maneuvering  and  not  reactivated  until  the  moment 
the  subject  pilot  took  over  control  [Raj,  1996].  Combining  audio  and  tactile  data  presented  adequate  information 
to  the  blindfolded  pilot  to  perform  GCA’s  down  to  minimum  altitude  or  complete  a  3  G  inside  loop  with  no 
more  than  one  degree  oscillation  in  roll. 

The  third  (hover)  flight  test  demonstrated  integration  of  normal  visual  flight  information  with  audio  cueing 
and  TSAS.  Pilots  in  this  test  flight  series  initially  commented  that  TSAS  appeared  to  increase  their  workload  as 
the  tactors  seemed  to  fire  in  constantly  changing  directions.  However,  once  the  pilots  saw  how  much  more 
accurate  their  hover  station  keeping  was,  they  began  to  trust  the  system  and  respond  quickly  to  the  cues.  Rapid 
response  to  the  cues  reduced  oscillation  around  a  given  hover  point  and  reduced  the  number  of  times  the  tactors 
actually  fired  during  a  given  maneuver  or  time  interval.  When  the  pilots  were  utilizing  the  increased  precision  of 
the  system  this  way,  they  reported  that  their  perceived  workload  decreased,  at  the  same  time  their  maneuver 
accuracy  increased.  This  training  was  rapid  since  this  change  in  workload  and  accuracy  typically  occurred  by 
the  third  attempt  at  a  particular  maneuver.  Figure  6  depicts  a  typical  simulated  shipboard  landing  from  the 
TSAS  hover  flight  test.  The  pilot  is  wearing  the  foggles  and  performs  the  task  of  decelerating  to  an  out  of 
ground  effect  (OGE)  hover  to  the  left  of  the  desired  landing  zone  (LZ).  Once  stabilized,  the  pilot  descends  to  in 
ground  effect  (1GE)  hover,  then  translates  to  the  right  until  over  the  LZ,  and  then  descends  to  land.  The 
histogram  shows  how  often  each  tactor  fired  during  the  maneuver  (for  the  TSAS  active  case).  Little  tactor 
activity  is  seen  aside  from  the  left  and  right  tactors.  This  manifests  also  in  the  velocity  plot  as  there  is  very  little 
drift  (graph  centered  about  zero  velocity),  except  to  the  right  when  translating  over  to  the  LZ.  Without  TSAS, 
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the  velocity  plot  shows  a  much  more  erratic  pattern  and  the  position  plots  demonstrate  decreased  precision. 
These  pilots  were  successfully  able  to  integrate  visual  instruments,  audio  cues  and  the  tactile  display  to  expand 
their  mission  capabilities. 


Simulated  Shipboard  Landing 


Isometric  View 


Northing  (meters) 


Easting  (meters) 


Histogram  of  Tactor  Activation 

F  250 


Overhead  View 


-20  -10  o  10  2n 

Easting  (meters) 


-  TSAS  On 


-  TSAS  off 


Figure  6:  Simulated  shipboard  landing  with  foggles,  with  and  without  TSAS. 


The  underwater  Field  test  successfully  demonstrated  improved  navigational  ability  in  poor  visibility  conditions. 
One  diver  commented  that  the  system  allowed  the  diver  “use  of  his  eyes  so  he  can  avoid  running  into  mines  or 
obstacles.”  Most  commented  that  the  system  was  intuitive  (“easy  to  learn”)  and  that  it  w'ould  easily  integrate  into 
their  operations  [Coastal  Systems  Station,  1997]. 

Lastly,  the  simulation  of  an  EVA  astronaut  rendezvous  showed  significant  improvement  in  ability  to 
perform  the  task  (both  in  reaction  time  and  time  to  complete  the  task).  Controlling  a  six  degree  of  freedom  EVA 
astronaut  model  with  a  six  degree  of  freedom  force  sensing  hand  controller  proved  difficult,  but  not  impossible. 
Subjects  often  lost  control  of  the  model  during  maneuvering  due  to  this  novel  input  device;  such  runs  were 
recorded  as  a  failure.  Figure  7  depicts  the  number  of  failures  encountered  for  each  of  the  four  scenarios,  and 
demonstrates  the  decrease  in  failures  seen  when  TSAS  is  integrated  with  the  visual  simulation.  This  graph  also 
shows  that  while  the  task  difficulty  increases  the  number  of  failures  decrease.  Since  the  harder  scenarios  require  a 
maneuver  to  initially  find  the  target,  subjects  may  be  more  cautious  with  their  inputs.  The  first  (easiest)  scenario, 
however,  has  the  target  in  sight  at  the  onset,  which  may  encourage  a  more  heavy  handed  approach  These  results 
demonstrate  that  tactile  cues  can  help  maintain  SA  and  control  of  a  maneuvering  platform  despite  the  use  of 
difficult  and  fatiguing  control  systems 
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Figure  7:  Failures  vs.  navigation  task,  with  and  without  TSAS. 

Conclusions.  The  TSAS  program  has  shown  that  tactile  cues  can  integrate  into  existing  situation  displays  and  can 
function  even  in  the  absence  of  other  SO  information.  Future  cockpit  and  other  platform  control  systems  will  no 
doubt  include  more  elaborate  SA  displays  [Joint  Advanced  Strike  Technologies,  19%]  that  would  benefit  from 
tactile  interfaces  to  provide  a  true  multimodal  display.  In  such  a  system,  the  effects  of  the  zone  of  confusion 
associated  with  3D  audio  could  be  mitigated  by  providing  additional  tactile  data  to  aid  in  localization  in  this  zone. 
Tactile  cues  may  be  used  to  direct  an  operator  toward  an  area  of  interest  outside  the  field  of  view  of  head  mounted 
displays  or  NVG's.  Likewise,  other  visually  demanding  displays  could  utilize  tactual  signals  to  steer  attention 
toward  critical  sections  of  the  display.  In  the  most  intuitive  form,  as  a  spatial  orientation  instrument,  TSAS  displays 
will  allow  a  pilot  to  focus  more  attention  on  other  mission  critical  tasks,  such  as  navigation  control  (as  seen  in  the 
EVA  simulation),  or  concentrating  on  a  map  or  approach  plate,  without  losing  SA.  In  multiship  operations,  TSAS 
could  be  used  to  provide  spacing  information  to  each  element  of  the  formation  with  proximity  warning  to  prevent 
collisions.  These  and  other  future  applications  would  improve  operational  capability  by  allowing  safe  flight  into 
more  hazardous  locations  with  fewer  meteorological  restrictions. 
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The  purpose  of  this  research  is  to  investigate  the  use  of  computational  models  of  human 
performance  to  design  and  evaluate  information  displays  in  military  cockpits.  The  focus  of  the  current 
work  is  using  computational  modeling  to  evaluate  auditory  interfaces.  The  approach  includes  four  steps: 
I)  develop  auditory  interface  designs;  2)  augment  a  computational  model  called  Executive-Process/ 
Interactive  Control  (EPIC)  to  incorporate  the  human  performance  effects  of  these  interface  designs;  3) 
collect  human  performance  data;  and  4)  evaluate  the  accuracy  of  the  model  against  the  empirical  data. 
Work  at  the  Naval  Research  Laboratory  (NRL)  includes  the  development  of  interface  designs  and  the 
collection  of  human  performance  data  David  Kieras  and  David  Meyer  are  doing  the  EPIC  modeling  at 
the  University  of  Michigan.  This  paper  reports  some  of  the  results  of  the  empirical  studies  at  NRL. 

Computational  modeling  is  emerging  as  a  potential  method  to  do  early  system  design.  Initial 
work  in  this  area  focused  on  single-task,  office  workstation  domains,  but  recently  Kieras  and  Meyer 
(1995)  have  been  successful  in  modeling  dual  task  performance  in  a  low  fidelity  cockpit  domain.  Their 
work  has  shown  that  control  of  peripheral  resources,  particularly  eye  movement,  is  the  dominant  factor  in 
mediating  performance.  This  focus  on  eye  movement  as  a  mediator  of  performance  is  consistent  with  the 
perspective  of  the  aviation  community,  which  has  long  recognized  the  importance  of  eye  movement. 
Basic  Instrument  Flight  training  manuals  include  sections  on  visual  scanning,  emphasizing  the 
importance  of  efficient  scan  patterns.  For  example,  the  Navy’s  flight  training  instruction  manual  for  basic 
instruments  includes  the  following  paragraph: 

”ln  order  to  determine  the  aircraft's  attitude  quickly  and  effectively,  you  must  know  what 
instruments  to  scan  for  a  particular  maneuver.  The  following  section  lists  the  correct 
instruments  or  scan  "pattern"  for  every  situation.  It  is  mandatory  that  the  student  commit  these 
patterns  to  memory."  (Naval  Air  Training  Command,  1989,  p,  16) 

The  research  reported  here  is  based  upon  previous  work  at  NRL  (hereafter  referred  to  as  the  '92 
study)  on  the  design  of  adaptive  automation  interfaces.  Balias,  Heitmeyer  and  Perez  (1992)  found  that 
certain  interface  features  mitigated  a  transient  automation  deficit.  This  effect  is  seen  in  longer  reaction 
times  on  the  first  few  responses  upon  resuming  a  task  that  has  been  previously  automated.  The  effect  is 
short-lived,  and  has  not  been  found  in  studies  that  examine  average  performance  over  several  minutes. 
For  example,  Parasuraman,  Hilbum,  Molloy  and  Singh  (1991)  reported  that  there  was  no  evidence  of 
automation  cost  in  their  studies  of  adaptive  automation.  However,  their  dependent  variable  was  average 
performance  over  a  10-minute  block  of  trials.  In  addition,  the  task  demand  was  low  in  the  phase  after 
automation,  under  the  assumption  that  a  task  would  be  returned  to  the  person  only  under  conditions  of 
low  performance  demand.  The  '92  study  found  the  automation  deficit  effect  on  responses  in  the  first  few 
seconds,  under  conditions  of  heavy  demand. 

The  '92  study  used  a  low  fidelity  cockpit  simulation  in  which  the  subject  performed  two  tasks, 
tracking  and  tactical  assessment  Kieras  and  Meyer  have  modeled  the  human  performance  of  both  tasks 
in  that  study,  including  the  automation  deficit  effects.  Empirical  predictions  can  be  made  from  their 
model,  and  this  paper  is  a  report  about  two  of  these  predictions.  The  first  prediction  is  about  automation 
deficit  Their  detailed  model  generated  an  explanation  for  this  effect:  upon  resuming  the  task,  the  subjects 
randomly  scan  the  tactical  assessment  window  and  handle  the  objects  in  a  random  order.  This  produces 
longer  reaction  times  on  the  events  that  should  be  handled  first.  If  the  subjects  have  been  performing  the 
task,  they  know  about  the  developing  pattern  and  handle  the  objects  in  the  proper  order,  producing 
shorter  reaction  times.  Based  on  this  explanation,  it  was  predicted  that  the  effect  would  not  occur  if  the 
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subjects  began  the  task  with  a  low  level  of  demand.  To  test  this  prediction,  the  *92  study  was  repeated 
with  two  scenarios:  one  similar  to  what  was  used  before  (i.e.,  with  the  heavy  demand  when  the  task  is 
resumed),  and  a  scenario  with  a  moderate  task  demand. 

The  second  prediction  is  about  performance  in  the  tracking  task  when  kevpresses  are  made  in 
the  tactical  assessment  task.  The  EPIC  model  assumes  that  the  tracking  task  is  momentarily  suspended 
when  the  eyes  move  into  the  tactical  assessment  task  window,  as  well  as  when  the  response  on  the 
tactical  task  is  executed.  To  evaluate  this  assumption,  the  software  was  modified  to  record  more  details 
of  tracking  performance,  and  an  analysis  was  conducted  of  the  joystick  holds  that  occur  when  keypresses 
are  made. 


Method 

Experimental  Tasks 

Subjects  performed  two  tasks,  a  pursuit  tracking  task  and  a  tactical  assessment  task.  A  summary 
of  the  tasks  is  shown  in  Figure  1.  To  establish  a  setting  for  adaptive  automation,  the  difficulty  of  the 
tracking  task  was  modified  in  phases  throughout  the  experiment.  During  moderate  difficulty  phases,  the 
subject  performed  both  the  tracking  task  and  the  tactical  assessment  task.  When  the  tracking  difficulty 
was  increased,  the  tactical  assessment  task  was  automated,  and  the  subject  performed  the  tracking  task 
only.  The  display  screen  used  in  the  experiment  was  partitioned  into  two  windows,  one  for  the  tracking 
task,  the  other  for  the  tactical  assessment  task.  Changes  in  the  automation  of  the  tactical  assessment  task 
were  signaled  in  two  modalities.  A  beep  occurred  at  each  change  and  a  border  was  placed  around  the 
window  when  the  task  was  performed  manually.  The  color  of  this  border  matched  the  border  of  the 
tracking  window  so  that  the  tasks  would  be  integrated  while  both  were  in  the  manual  mode. 


Tactical  assessment 

Display:  Graphical 

Input  Keypad 

User  involvment.  Intermittent 


Tracking 

Difficulty  Two  levels 

Inpul  Joystick 

User  Involvment:  Continuous 


Automation  Logic 

L  H  L  H  L  II  L 

L  =  moderate,  1 1  =  high 

Tactical  Mode:  M  A  M  A  M  A  M 

M  *  manual,  A  *  automatic 


Figure  I .  Interfaces  for  the  tactical  assessment  task  and  the  tracking  task  and  a 
summary  of  the  interface  properties  and  automation  logic. 

The  tracking  task  simulated  air-to-air  targeting  of  an  enemy  aircraft  using  a  gun  sight  similar  to 
the  piper  and  reticle  on  a  typical  head-up  display.  The  target  on  the  display  was  a  graphical 
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representation  of  an  enemy  aircraft.  The  target's  driving  function  was  the  sum  of  nine  nonharmonic 
sinusoids  (.02,  .03,  .07,  13,  .23,  .41,  .83,  1.51,  and  3.07  Hz)  with  randomly  determined  starting  phases. 
The  amplitudes  of  these  components  were  varied  to  produce  two  levels  of  tracking  difficulty.  The 
amplitudes  for  the  “less  difficult'’  tracking  were  flat  up  to  a  cutoff  frequency  of  .07  Hz  and  reduced  in 
amplitude  3  dB/octave  above  this  frequency.  The  “difficult”  function  was  flat  up  to  a  cutoft  frequency  of 
.23  Hz  and  reduced  in  amplitude  3  dB/octave  above  this  frequency.  The  target  position  was  updated 
every  83  ms  and  the  joystick  position  was  sampled  at  the  same  rate.  The  tracking  device  was  a  self¬ 
centering,  displacement  joystick.  The  control  dynamics  were  a  25%/75%  mixture  of  rate  and 
acceleration.  A  continuous  record  of  the  joystick,  target  and  piper  position  was  recorded. 

For  the  tactical  task,  the  subjects  assessed  three  types  of  simulated  tracks - fighters,  aircraft, 

and  ground-based  missiles.  The  tracks  appeared  on  the  screen  as  possible  threats  using  black  color 
coding,  and  as  they  got  closer  to  the  ownship  (the  symbol  for  the  aircraft  the  pilot  was  in),  they  were 
designated  as  neutral,  hostile,  or  unknown,  using  blue,  red,  and  amber  color  coding,  respectively.  The 
subjects  were  told  that  simulated  sensor  systems  were  assigning  these  designations  The  subjects  were 
required  to  make  one  of  two  decisions,  confirm  or  classify,  on  each  track,  if  the  system  designated  a 
track  as  neutral  or  hostile  (i.e.,  the  track  was  colored  blue  or  red),  the  subject  had  to  confirm  the 
designation  by  picking  the  track  and  then  indicating  the  proper  designation,  i.e.,  neutral  for  blue  tracks 
and  hostile  for  red  tracks.  Thus,  confirm  decisions  only  required  the  subject  to  discriminate  colors.  If  the 
system  designated  the  track  as  unknown  (i.e,,  the  track  was  colored  amber),  the  subject  had  to  classify 
the  track  as  hostile  or  neutral  based  on  its  behavior.  Table  1  shows  the  rules  for  making  this  decision. 


Table  1 .  Rules  for  Tactical  Assessment  of  Tracks 


Track  Class  Hostile  Neutral 

Fighter 

On  approach  bearing 

Bearing  away 

Airplane 

Air  speed  ~  800 

Air  speed  -  300 

Missile  site 

Within  threat  range 

Outside  threat  range 

To  classify  the  amber  tracks,  the  subject  needed  to  monitor  bearing  for  fighters,  speed  for 
aircraft,  and  projected  lateral  distance  for  ground  missile  threats.  The  responses  were  timed  and  analyzed 
to  produce  measures  of  accuracy  and  response  time.  The  subject  had  a  response  interval  of  10  seconds  to 
make  the  assessment  response. 

The  tactical  interface  (Figure  1)  simulated  a  radar  display  with  continuously  moving  symbols 
representing  the  tracks.  The  bottom  portion  echoed  the  subject's  keypresses.  The  subject  used  a  keypad 
to  insert  a  confirm  or  classify  decision.  For  each  decision,  two  keypresses  were  required,  one  designating 
hostile  or  neutral,  the  second  indicating  the  track  number. 

Training  and  Scenarios 

The  subjects  were  tested  in  three  1-hour  sessions.  The  first  two  sessions  were  used  for  training 
and  collection  of  single  and  dual  task  performance  data.  Training  time  was  doubled  over  the  '92  study. 
In  the  third  session,  they  were  tested  in  two  25  minute  sessions  with  intermittent  automation  of  the 
tactical  task.  Two  scenarios  with  different  task  demand  were  used  in  these  sessions.  In  each  of  these 
sessions,  they  started  with  a  dual-task  phase,  performing  both  tasks.  They  then  went  through  6 
automation-manual  cycles  as  illustrated  in  Figure  1.  The  duration  of  the  automation  and  manual  phases 
varied  between  105  and  135  seconds.  When  the  tactical  task  was  automated,  the  tracking  task  increased 
in  difficulty.  During  the  intermittent  automation  sessions,  the  scenarios  produced  138  tracks  that  had  to 
be  confirmed  or  classified,  for  an  average  event  rate  of  one  per  12  seconds.  About  75  of  these  tracks 
were  presented  for  manual  assessment.  Two  scenarios  were  used:  one  in  which  the  tracks  occurred  at  a 
relatively  steady  pace  (Level  scenario)  and  one  in  which  the  tracks  occurred  at  a  variable  rate  (Wax  & 
Wane).  The  latter  scenario  was  used  in  the  '92  study  and  demanded  several  decisions  at  the  beginning  of 
each  manual  phase.  This  meant  that  the  transition  from  automated  to  manual  operation  occurred  at  a 
period  of  high  task  demand.  This  demand  was  repeated  later  in  the  manual  period  to  obtain  comparison 
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data.  Specifically,  the  properties  of  stimulus  seven  matched  the  first  stimulus  in  the  manual  phase,  and 
the  inter-stimulus  intervals  between  stimuli  seven,  eight  and  nine  matched  the  intervals  between  stimuli 
one,  two  and  three.  The  evaluation  of  an  automation  deficit  has  been  comparing  performances  on  stimuli 
one  and  seven. 

Results  and  Discussion 

The  average  time  to  assess  the  tactical  tracks  in  the  manual  phases  are  shown  in  Figure  2,  for 
each  of  the  scenarios.  For  comparison  purposes,  the  average  times  from  the  '92  study  for  the  same  type 
of  tactical  interface  are  also  shown.  The  ’92  study  evaluated  four  interfaces  for  the  tactical  assessment 
task.  Only  the  graphical-keypad  interface  was  used  in  this  research  The  ’92  study  only  used  a  wax  and 
wane  scenario. 

The  automation  deficit  effect  was  computed  by  taking  the  difference  between  the  first  and 
seventh  responses.  In  the  '92  study,  this  difference  was  about  1500  ms;  in  the  current  study  this 
difference  was  about  900  ms.  The  Wax  &  Wane  results  in  the  current  study  also  show  longer  times  for 
events  following  stimulus  seven.  These  results  are  consistent  with  a  Psychological  Refractory  Period 
(PRP)  since  stimuli  eight  and  nine  are  presented  with  short  int^r-stimulus  intervals.  But  the  increased 
response  times  for  these  stimuli  over  the  '92  study  is  confusing.  Recent  modeling  (Kieras,  1997)  of  these 
data  suggests  that  the  current  subjects  were  performing  nearly  as  well  as  they  could  on  the  first  two 
stimuli,  but  performing  at  an  average  level  on  the  eighth  stimulus.  If  this  is  the  case,  it  would  explain 
why  the  automation  deficit  effect  is  less  in  the  current  study  (because  of  better  than  expected 
performance  on  the  first  stimulus)  and  why  the  PRP  effect  is  greater  on  event  eight  in  the  current  study 
(average  performance  on  stimulus  eight). 
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Figure  2.  Response  times  to  classify  stimuli  during  the  manual  phase  of  the  tactical  assessment 
task.  Data  are  for  two  scenarios  in  a  ’97  study,  and  one  scenario  in  '92. 

As  predicted  by  the  EPIC  computational  modeling,  the  deficit  did  not  occur  with  a  level 
scenario,  which  eliminated  the  heavy  demand  at  the  onset  of  the  manual  phase.  The  level  scenario  also 
did  not  present  a  stimulus  to  be  classified  simultaneous  with  the  end  of  automation  (and  onset  of  manual 
tactical  assessment).  These  results  imply  that  the  deficit  effect  will  occur  when  either  one  or  both  of  two 
conditions  are  present:  I)  the  first  event  occurs  at  or  shortly  after  the  beginning  of  the  manual  phase;  2)  a 
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heavy  task  demand  is  present  at  the  beginning  of  the  manual  phase.  Further  studies  eould  investigate 
these  two  possibilities  in  detail 

The  second  prediction  being  addressed  is  that  the  tracking  task  is  suspended  while  a  person 
scans  the  tactical  assessment  display  and  while  keypresses  are  made  for  the  tactical  assessment.  These 
predictions  follow  from  the  task  switching  logic  in  the  EPIC  model  for  this  dual  task.  To  investigate  this 
prediction,  successive  positions  of  the  joystick  were  compared  to  locate  sMck  holds  The  length  of  these 
holds  was  tallied  and  plotted  in  a  frequency  distribution.  The  result  is  an  example  of  Zipfs  law'  (Figure 
3).  With  the  distribution  of  holds  known  for  the  tracking  record  as  a  whole,  a  second  distribution  was 
tallied  which  includes  onl\  holds  that  began  when  keypresses  were  made  To  derive  this  distribution,  the 
keypress  times  were  used  as  array  pointers  into  the  tracking  record.  This  focused  the  comparison  of 
successive  joystick  positions  onto  times  when  keypresses  were  initiated  If  a  hold  did  not  occur  at  a 
particular  keypress  time,  then  the  comparison  was  stopped  for  that  location  in  the  record,  and  resumed  at 
the  next  keypress  time.  The  analysis  was  designed  so  that  a  long  joystick  hold  that  spanned  more  than 
one  keypress  would  onlv  be  tallied  once. 

The  two  distributions  are  shown  in  Figure  3  for  three  scenarios:  continuous  dual  task  with  no 
intermittent  automation,  and  the  two  scenarios  with  intermittent  automation.  For  all  three  scenarios,  the 
proportion  of  holds  is  greater  at  keypresses.  By  implication,  the  proportion  of  no  holds,  which  is  not 
plotted  in  these  figures  because  it  is  in  the  range  of  .6-. 8,  is  greater  for  the  overall  distribution.  The 
difference  in  the  distributions  occurs  for  holds  up  to  a  second  in  length. 

Holds  in  Continuous  Dual  Task  Holds  jn  Leve)  scenario 


Holds  in  Wax  And  Wane  Scenario 


Holds(1/12s) 


Figure  3=  Tracking  holds  throughout  the  tracking  record  and  at  keypresses  in  three  scenarios. 


89 


These  results  support  the  assumption  made  in  the  EPIC  model.  However,  they  do  not  tell  how 
often  tracking  is  suspended  at  keypresses,  and  do  not  address  the  suspension  of  tracking  when  eye 
movement  is  out  of  the  tracking  window  and  into  the  tactical  window.  Further  analyses  of  the  tracking 
data  will  irvestigatc  the  first  issue. 

These  results  generally  support  the  predictions  from  computational.  Two  predictions  that  came 
from  the  modeling  were  generally  supported  in  subsequent  empirical  studies.  Of  course,  these 
predictions  might  have  been  made  in  the  absence  of  the  modeling  effort,  which  is  difficult  and  expensive. 
Certainly  tracking  holds  have  been  a  topic  in  the  manual  control  literature.  However,  the  computational 
modeling  can  focus  the  predictions  and  have  both  a  strategic  (suggest  general  issues)  and  a  tactical 
(suggest  specific  experimental  design  modifications)  on  empirical  research.  These  benefits  are  in 
addition  to  the  early  impacts  on  system  design  impacts  of  a  computational  model. 
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Applications  of  3-D  Auditory  Displays  in  Threat  Avoidance, 
Collision  Avoidance ,  and  Target  Cueing 


Richard  L.  McKinley  and  W.  Todd  Nelson 

Air  Force  Research  Laboratory 
Wright-Patterson  Air  Force  Base,  Ohio 

Almost  everyone  uses  spatial  (3-D)  auditory  information  in  everyday  life.  When  a  person  is  standing 
on  a  street  comer,  they  know  the  direction  and  approximate  distance  of  an  approaching  car  or  bicycle,  which  is 
outside  their  current  field  of  view  by  using  their  auditory  localization  capabilities  The  goal  of  the  Air  Force 
Research  Laboratory’s  3-D  audio  display  program  is  to  provide  this  type  of  sensory  information  over 
headphones  in  the  cockpit  for  threat  avoidance,  collision  avoidance,  and  target  cueing. 

The  in-house  experiments  in  aurally  guided  visual  search  have  demonstrated  an  approximate  2X 
improvement  in  target  acquisition  times,  2X  improvement  in  target  detection  range,  and  2X  to  10X 
improvement  in  locating  targets  in  clutter  or  decoys. 

Most  of  these  laboratory  findings  have  been  verified  in  flight  tests  on  NASA  Lewis  Research  Center’s 
OV-IO  flight  test  aircraft  and  Naval  Air  Warfare  Center’s  AV-8B  Harrier  flight  test  aircraft.  The  aircraft 
objective  results  mirrored  the  laboratory  results  and  the  subjective  comments  from  the  flight  test  pilots  described 
significant  improvements  in  situational  awareness  and  attack  capability  with  reductions  in  workload 

These  findings  have  direct  application  to  USAF  and  Navy  combat  aircraft  operations  as  well  as 
applications  to  civilian  aircraft  operations. 


(Reprint  of  executive  summary;  formal  paper  not  available.) 
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Naval  Situational  Awareness  Enhancements  within  the 
Maritime  Avionics  Subsystems  and  Technologies  (MAST): 
Advanced  Graphics  and  Data  Fusion 


Floyd  Adagio 

Cambridge  Research  Associates, 

1430  Spring  Hill  Road,  McLean,  VA,  22102  USA, 

(703)  790-0505  ext.  7130,  email:  adagio@cambridge.com. 
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Prepared  for  the  3rd  Annual  Symposium  on  Situational  Awareness,  under  the  Topic  of  Spatial  Awareness 
Interface  Considerations  in  the  Tactical  Air  Environment,  Patuxent  River  MD,  June,  1998 

Summary:  Various  levels  of  the  military  command  infrastructure  require  visualization  of  data  and  fusion  of 
tactical  and  strategic  information.  Specifically,  visually  oriented  displays  can  provide  intuitive,  readily  under¬ 
standable  information  which  is  easily  interpreted  and  acted  upon.  Visual  systems  provide  a  variety  of  data  and 
information  which  include  key  elements  such  as  maps,  terrain  elevation,  imagery,  iconic,  symbology,  and/or  text. 
Use  of  these  information  forms  must  be  specific  to  individual  missions  yet  consistent  to  all.  Consistency  among 
and  between  command  levels  is  critical  to  support  coordinated  planning,  execution,  and  after-action  activities. 

Information  forms  and  presentation  to  a  user  are  constrained  by  system  and  cost  considerations  This 
presentation  and  associated  manuscript  provide  a  discussion  of  application  of  virtual,  visualization  environments 
within  a  hierarchical  requirements  structure  based  on  warfighting  functional  requirements.  Additionally, 
functional  requirements  are  related  to  system  characteristics  and  utility  to  airborne  applications  are  also  discussed. 

Introduction:  The  ability  to  convey  information  and  to  exploit  it  is  an  age  old  issue.  Plato  referred  to  the  purist  of 
information  as  a  “form”1.  He  also  discussed  mental  processes  which  are  appropriate  to  military  visualization 
within  a  continuum  of  mental  processes  which  include  understanding,  reasoning,  opinion,  and  imagination.  The 
philosopher  discussed  his  notion  of  forms  as  the  highest  level  of  knowledge.  In  the  present  as  well  as  the  past,  the 
ability  to  impart  the  highest  level  of  knowledge  is  critical  to  the  human  endeavor  whether  in  war  or  peace.  Thus, 
the  objective  of  providing  the  perfect  information  should  strive  to  provide  understanding  by  intuitive  means  at  the 
form  level  of  the  knowledge  continuum 

Today’s  visualization  of  military'  forms  presents  unique  challenges  and  perplexing  technological  trade¬ 
offs.  Achieving  the  right  look  or  feel  for  combat  data  is  difficult.  A  gap  in  communicatioa'  interpretation  exists 
between  users  and  developers.  Decision  makers,  who  are  of  the  slide  rule  generation,  are  often  determining 
system  needs  for  operators  who  grew'  up  using  video  games  and  personal  computers. 

Discussion.  Visualization  requirements  span  a  continuum  from  simple  line  depiction  to  fusion  of  multiple 
sources  of  data  and  rendering  of  high  fidelity,  geo-specific  perspective  scenes  in  real-time.  This  range  of 
capabilities  has  been  employed  to  support  mission  planning,  rehearsal,  execution,  debrief;  intelligence  efforts;  and 
modeling/simulation  efforts. 

As  provided  within  US  Defense  Science  and  Technology  Guidance  for  Joint  DOD  applications  several 
areas  are  impacted  by  visualization  enhancements.  Specifically,  Command,  Control,  Communications,  and 
Computers  (C*)  (Provide  common,  accurate,  mission-tailored  picture  to  all  warfighters;  Develop  ability  to  “learn" 
from  users,  and  improve  visualization);  Intelligence,  Surveillance,  Reconnaissance  (ISR)  (Improve  situational 
awareness,  manage  information  and  decision  aids);  Manpower  /  Personnel:  (Determine  optimum  hu¬ 
man/computer  tasking,  improve  decision-making,  reduce  user  workload,  and  reduce  crew  /  manning  require¬ 
ments).  These  capabilities  will  foster  performance  improvements  and  manpower  reduction  by  improving 
warfighter  performance,  and  reduce  workload  by  tailoring  displays  to  user  requirements. 

Consistent  and  repeatable  visualization  capability  are  critical  throughout  and  among  the  command  lev¬ 
els.  Example  of  advanced  applications  for  concurrent  visualization  at  multiple  levels  include  recent  activities  for 
Bosnian  Operations:  planning  and  rehearsal  during  Deliberate  Force  Operations,  use  of  virtual  environments  for 
the  Bosnian  peace  talks.  Exploitation  within  Joint  Strike  Fighter  (JSF)  development  program,  Investigation  of  US 
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Secretary  of  Commerce  Brown’s  Crash,  and  Convey  data  to  US  National  Command  Authority.  Simulated 
environments  offer  the  opportunity  to  “see  into  the  future”  and  allow  operators  to  expand  their  understanding  of 
future  events  while  expanding  situational  awareness.  These  virtual  environments  also  provide  user  augmented 
reality  with  actual  situations  which  may  have  limited  information  resources.  An  example  is  evident  in  airborne 
applications  to  enable  flexible  targeting  and  retasking  as  well  as  a  context  for  off-board  information  fusion. 

The  Tactical  Moving  Map  Capability  (TAMMAC)  is  planned  for  F-18,  AV-8B,  and  AH-1 W  with  po¬ 
tential  applications  to  the  V-22,  UI1-1N,  F- 14,  P-3C,  and  H-60. 2D  map  requirements  are  included  in  Table  1 . 


Table  1 . 2D  Requirements  Based  on  the  US  Navy  Tactical  Air  Moving  Map  Capability.2 


Change  of  map  scales  and  features  in  real-time 

Mag/Zoom  from  1  to  1  up 

Coverage  area  in  standard  scales 
(ex.  1  inch  to  5  million  meters) 

Programmable  color  palettes 

Variable  map  scales  between  2.5  and  250  nm 

Standard  symbol  generation 

Map  orientation  north-up  or  heading-up 

Dynamic  overlays 

Center/de-center 

Dataframe  rates 

Sun/moon  shading 

Slewing 

3D  perspective,  scene  generation  requirements  can  be  found  in  Table  2.  Figure  1  provides  examples  of 
desire  2D  rendering  capabilities  for  the  TAMMAC.  Planned  TAMMAC  improvement  included  implementation 
of  3D  functions.  3D  perspective  scene  generation  needs  were  assessed  in  terms  of  user  suggested  enhancements  to 
Bosnian  prototype  systems  and  desire  JSF  capabilities 

Table  2  provides  a  list  of  3D,  scene  perspective  capabilities  based  on  situational  awareness  needs  related 
to  the  JSF  STT  framework.  Users  have  also  asked  that  future  visualization  to  incorporate  other  sensor  imagery, 
Provide  simulations  of  sensor  and  performance  predictions.  Extrude  objects  and  buildings,  Provide  aircraft 
appropriate  flight  dynamics,  Add  weapon  simulations.  Display  threats  dynamic/accurately,  Provide  interface  to 
mission  planning  systems,  and  Permit  user  to  retrieve  coordinates  from  scene. 


Table  2.  3D  Requirements  Based  on  the  JSF  Virtual  Ground  Map  Functionality 


Multi-source/spectral  data 

Threat  overlays 

Mutli-source/resolution  elevation 

Updated  and  real-time  threat  overlays 

Geo- located/specific  imagery  and  elevation 

Multiple  fields  of  view 

Display  of  vector  products/data 

Degrees  of  freedom  view  angle  control 

Annotations  and  pointers 

Remote  and  slewable  eye  point 
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Figure  1.  Current  2D  Map  Capabilities.  Example  illustrates  map  data  over  Sarajevo 
and  includes  tactical  overlays. 


Figure  2  provides  a  notional  view  of  a  perspective  scene  capability  using  photo-based  imagery  and  ele¬ 
vation  data  and  appropriate  strike  symbology.  This  includes  threat  domes,  target  locations,  collateral  damage 
warning,  approach  restrictions,  sensor  updates  and  weapons  effects.  The  2D  information  portrayed  in  Figure  1  is 
an  example  of  a  display  which  requires  interpretation  by  the  user. 


The  information  in  Figure  2  provides  intuitive  graphics  in  mission  context  for  a  strike  platform. 
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Figure  2.  Futuristic  3D  Perspective  Capabilities.  Example  shows  unclassified  US  Geological  Survey 
imagery  &  elevation  data  over  the  Naval  Air  Station  China  Lake,  CA 


Additionally,  the  growth  of  communications  and  display  techniques  provides  an  opportunity  to  update 
this  information  within  the  mission  timeline  providing  expanded  sensor  data  to  the  warrior.  The  intent  is  to 
provide  Common  Battlefield  Awareness  which  provides  command,  control,  and  operational  data  to  user  within 
his/her  mission  context. 

As  depicted  in  Figure  3,  the  utility  of  this  data  can  be  seen  at  various  levels  of  the  command  infrastruc¬ 
ture.  At  the  command  center  levels  mission  preview  and  rehearsal,  fusion  of  intelligence  and  threat  data  can  be 
performed  providing  enhanced  command  and  control  using  intuitive  display. 
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Figure  3.  Warfighting  Visualization  Tasks  at  Various  Levels  of  the  Command  Infrastructure. 

Additionally,  what  if  scenarios  can  be  played  using  advanced  modeling,  simulation,  and  analysis  tools 
which  can  provide  campaign  level  analysis?  Tactical  information  could  be  fused  into  such  pictures  for  real-time 
operational  use  of  data  from  global  positioning,  imagery,  and  Direct  Broadcast  Systems.  Data  can  also  overlay 
and  update  via  Intel  sources.  These  command  level  activities  can  be  used  to  provide  more  specific  strategic 
direction  to  operational  levels  for  squadron  level  planning.  Future  techniques  would  exploit  “smart”  applications 
to  parse  data  appropriate  to  operational  levels  from  the  strategic  command  levels.  Additionally,  reach  back 
capabilities  for  lower  command  levels  could  provide  greater  insight  into  great  campaign  level  goals. 

Operational  level  data  could  be  passed  further  to  individual  mission  areas  and  exploited  at  lower  levels 
by  individual  operators  within  a  specific  mission  context.  Integrated  tactical  displays  linked  to  information 
resources  at  unit  and  command  levels  could  provide  current  operational  information  beyond  that  available  today. 
Such  capabilities  could  enable  flexible  retargeting  and/or  replanning;  in-flight  updates  with  data  and  imagery;  and 
remote,  future  views  within  virtual  reality  concepts.  Operational  concepts  might  include  modes  which  include 
video  teleconferencing. 

At  the  operational  level,  virtual  walk-/fly-through  capabilities  would  allow  users  to  see  through  the  next 
hill  or  discuss  safe  transit  routes.  Interfaces  to  mission  planning  resources  could  provide  the  view  ahead  and 
support  positional  and  maneuver  control  or  electronic  gridding.  Visually  augmented  identification  of  friend  or  foe 
as  well  as  obstacle  identification  within  tactical  situation  displays  could  be  provided.  Opening  the  operators  line 
of  sight  by  projecting  an  outside  view  through  the  wall  of  an  amphibious  tank  or  submarine  could  provide  a  better 
understanding  of  the  world  around  and  object.  Underwater  Threats  Structure,  Obstructions,  and  Mines,  Beach 
Assault  Routes  could  be  displayed  to  support  these  users  needs. 

Two  important  principles  which  should  guided  future  visualization  developments  are:  (1)  an  emphasis 
on  using  real-world  photo-imagery  and  (2)  the  use  of  portable,  standardized  software  tools  that  do  not  require 
non-standard  hardware.  Additionally,  military  and  commercial  systems  which  support  visualization  should: 
use/display  real  world  data,  from  any  perspective,  and  in  any  desired  level  of  detail;  exploit  imagery  of  vary  ing 
source,  scale,  breadth,  or  resolution  from  satellites,  aerial  photographs,  and  other  sources  and  combine  into  a 
seamless  visual  scene;  and  correlate  to  real-world  coordinates,  other  data  (such  as  maps  and  cultural  features)  and 
combine  with  the  imagery  and  display  together. 
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Military  visualization  should  also  consider  the  opportunity  provided  by  commercial  imagery  sources 
Present  capabilities  include  use  of  defense,  Government,  and  commercial  providers.  However,  emerging  sources 
of  this  data  in  high  resolution  formats  should  be  seriously  considered.  These  resources  will  make  data  a  com¬ 
modity  rather  than  a  luxury'. 

Summary:  Future  visualization  techniques  for  military  users  include  requirements  to  provide  intuitive  two 
dimensional  and  three  dimensional  data  within  a  mission  context.  Graphics  techniques  presently  provide  area 
expanding  to  provide  consistency  within  military  command  levels  and  provide  greater  opportunity  within  the  C41 
context.  The  user  of  these  integrated  techniques  will  foster  the  development  of  enhanced  operational  concepts 
and  future  operations  which  improve  the  militaries  abilities  to  reach  strategic  and  operational  goals. 

1.  Plato  translated  by  G.M.  Grube,  Plato's  the  Republic,  (Hackett  Pub.  Co.;  Indianapolis,  ID;  400  BC 
(1974));  Pg.  164. 

2.  M.  Lohrenz,  et  al.,  TAMMAC  Digital  Map  Requirement  Study  in  support  of  Advanced  Cockpit 
Moving  Map  Display,  NRL  Formal  Report  No.  7441-96-9652,  (Stennis  Space  Center;  Stennis  MS);  Pub. 
Date  TBD;  Pg.  5 
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Physiological  Implications  for  a 
Helmet-Mounted  Display  Spatial  Awareness  Interface 


Carita  A.  DeVilbiss,  Ph.D. 
AF  Research  Laboratory 
Brooks  AFB,  TX 


William  R.  Ercoline,  M.S. 

Veda,  Inc. 

Brooks  AFB,  TX 


Advanced  display  concepts  and  rapidly  evolving  interface  technology  have  brought  to  the  forefront  the  possibility 
of  a  helmet  mounted  display  (HMD)  becoming  an  integral  part  of  the  tactical  aircraft  cockpit  by  providing  primary 
flight  information  in  addition  to  mission  specific  capabilities.  This  represents  a  radical  change  in  pilot-cockpit  interface. 
If  we  are  to  successfully  pursue  this  goal,  the  physiological  implications  of  employing  a  HMD  as  a  primary  spatial 
awareness  interface  must  be  addressed. 

Life  Support  Equipment:  The  first  major  consideration  addresses  the  complications  encountered  when  an  aircraft 
cockpit  display  is  located  not  on  the  cockpit  structure,  but  instead  attached  to  the  pilot's  head.  To  be  an  effective 
“primary  flight  reference,"  a  HMD  would  need  to  be  securely  in  place  for  the  duration  of  the  mission  The  HMD 
would  not  only  need  to  accommodate  a  wide  variety  of  face  structures,  but  also  it  must  integrate  with  life  support  and 
protective  equipment,  e.g.,  Combat  Edge  and  Laser  Eye  Protection  Because  of  the  criticality  of  a  primary'  flight 
reference,  the  issues  of  comfort,  stability,  helmet  mass,  neck  strain,  thermal  strain,  and  visual  problems  take  on 
increased  meaning. 

Workload:  Generally  speaking,  spatial  disorientation  mishaps  result  from  inattention,  distraction,  or  just  plain 
excessive  mental  workload.  The  increasingly  complex  cockpit  environment  continues  to  be  a  challenge.  HMDs  will 
exasperate  the  current  flight  duties,  An  effective  spatial  awareness  interface  must  provide  continuous  intuitive  spatial 
orientation  information  aimed  at  minimizing  workload.  However,  it  is  necessary  to  thoroughly  understand  system 
limitations  and  flight  task  requirements  to  develop  flight  symbology  that  will  reduce  the  causes  of  increased  workload. 

Human  Sensory  Systems:  Especially  in  the  tactical  air  environment,  perceived  spatial  awareness  develops  from 
both  visual  and  vestibular  stimuli  as  well  as  preconscious  orientation  percepts.  The  flyer’s  mental  model,  whether 
correct  or  incorrect,  is  an  integration  of  continuous  multisensory  inputs  to  the  brain.  The  phenomena  of  visual 
dominance,  vestibular  suppression,  and  vestibular  opportunism  are  central  to  an  accurate  mental  model 

Issues:  Traditional  laboratory  studies  have  primarily  focused  on  the  visual  aspects  without  incorporating  simulta¬ 
neous  vestibular  or  tactile  sensory  inputs.  With  the  increased  interest  in  expanding  mission  effectiveness  with  helmet- 
mounted  displays,  the  dynamics  of  head  movements  in  conjunction  with  aircraft  movements  may  increase  the  impact  of 
these  multisensory  signals.  A  major  task  of  the  Spatial  Disorientation  Countermeasures  Task  Group  is  to  quantify  the 
phy  siological  significance  of  these  human  sensory  inputs  as  they  pertain  to  operationally  relevant  metrics.  Issues  that 
will  be  addressed  with  our  facilities,  such  as  the  Advanced  Spatial  Disorientation  Demonstrator,  include  visual  conflicts 
between  flight  symbology  and  external  scenes;  increased  vestibular  components  and  erroneous  perceptions  resulting 
from  the  combined  head  and  aircraft  motion;  simulated  flight  motion  with  and  without  conflicting  visuals,  head 
position,  and  collimation;  and  effects  on  flight  performance  when  transitioning  between  HMD  symbology  and  either 
head-up  display  or  head-down  display  symbology.  The  current  study  quantifies  the  effects  of  helmet-mounted  display 
attitude  symbology  motion  cues  on  basic  flight  tasks,  e.g.,  unusual  attitude  recoveries  or  precision  instrument  control 
tasks.  Significant  findings  from  the  completed  series  of  studies  will  form  the  basis  for  future  inflight  validation. 


(Reprint  of  executive  summary';  formal  paper  not  available.) 
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The  Physiological  Aspects  of  Mishaps  in  the  Tactical  Environment 


Paul  A.  Dolinarand  Estrella  M.  Forster 

Naval  Air  Warfare  Center  Aircraft  Division 
Patuxent  River,  MD 

The  operational  capability  of  current  high  performance  tactical  aircraft  is  pushing  the  physical  and  cognitive 
limits  of  the  aircrew.  The  demanding  flight  envelope  of  future  aircraft  will  only  further  jeopardize  the  aircrew’s  ability 
to  effectively  operate  the  aircraft  and  complete  the  mission  with  increases  in  altitude,  speed,  load  factors  and  high 
acceleration  (G)  maneuvers  with  rapid  onset  rates. 

The  forces  the  pilot  experiences  stress  the  cardiovascular  system  resulting  in  pooling  of  blood  away  from  the 
head  towards  the  extremities  and  eventually  leads  to  loss  of  vision  and  acceleration-induced  loss  of  consciousness  (G- 
LOC).  In  addition,  current  research  has  identified  G-related  altered  states  of  awareness  (ASA),  where  the  pilot  is 
awake,  but  exhibits  amnesia,  facial  tremors,  hand  arm  flailing,  spatial  disorientation  (SD),  and  may  not  perform 
instructions  critical  to  mission  completion.  ASA  not  only  occurs  at  high  G,  but  is  related  to  the  level  and  duration  of  the 
G  exposures  throughout  the  mission  and  may  even  occur  after  the  G  exposure.  Furthermore,  as  G-related  ASA 
episodes  are  not  identified  as  loss  of  consciousness,  these  episodes  may  be  misclassified  as  “loss  of  situational  aware¬ 
ness”  or  pilot  error,  especially  if  the  incident  under  investigation  involved  low  G  level  excursions. 

The  current  methods  used  to  combat  such  physiologic  events  include  anti-G  suits  and  valves,  physically  de¬ 
manding  straining  maneuvers,  elegant  displays,  intelligent  software,  and  pilot  training.  However,  Class  A  Mishaps  are 
still  occurring  at  an  alarming  rate.  Analysis  of  official  Navy/Marine  and  Air  Force  data  will  demonstrate  G-LOC  and 
other  altered  states  of  awareness  remain  a  problem. 


(Reprint  of  executive  summary;  formal  paper  not  available.) 
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THE  TACTICAL  AIRCRAFT  MOVING  MAP  CAPABILITY 

(TAMMAC) 

DIGITAL  MAP  SYSTEM 


Douglas  C.  Williams 

Harris  Corporation  -  Government  Aerospace  Systems  Division 
Melbourne,  Florida 

Fhe  US  Navy  is  currently  sponsoring  the  Tactical  Aircraft  Moving  Map  Capability  (TAMMAC)  program  to  de¬ 
velop  a  standard  Digital  Map  System  (DMS)  to  be  used  in  a  variety  of  Navy  and  Marine  Corps  aircraft.  Digital  Map 
requirements  from  both  fixed  and  rotary  wing  platforms  were  combined  with  the  results  from  Human  Factors 
studies’* 2  to  arrive  at  the  desired  capabilities  that  shaped  the  requirements  for  the  TAMMAC  DMS.  The  resultant 
system  offers  a  significant  improvement  in  aircrew  situational  awareness,  particularly  during  low  level  and  nap  of  the 
earth  (NOE)  flight 

The  TAMMAC  system  consists  of  a  Digital  Map  System  (DMS)  and  an  associated  Advanced  Memory  Unit 
(AMU)  for  loading  of  mission  planning  data  and  logging  of  maintenance  data.  This  paper  describes  the  baseline  and 
growth  capabilities  of  the  TAMMAC  DMS  as  they  pertain  to  enhancing  situational  awareness, 

Tlie  first  platform  to  receive  the  new  TAMMAC  DMS  will  be  the  F/A-l  8  for  the  C,  D,  E,  and  F  versions.  Other 
aircraft  planning  to  utilize  the  TAMMAC  DMS  include  the  AV-8B,  AH-1Z,  UH-1 Y,  MV-22,  and  some  versions  of  the 
CH-60.  Undoubtedly,  once  the  DMS  has  been  proven,  other  platforms  will  be  included  as  well.  In  addition,  since  the 
TAMMAC  system  has  received  a  Joint  designation,  it  is  also  being  considered  for  use  on  Air  Force  C-130s  and  Army 
CH-47Ds.  For  many  of  these  aircraft,  the  addition  of  a  color  moving  digital  map  display  will  in  itself  be  a  significant 
improvement  in  situational  awareness  since  the  flight  crews  currently  use  paper  maps.  Other  aircraft  have  been  flying 
with  a  moving  map  capability  for  some  time  and  for  them,  the  TAMMAC  DMS  offers  improved  image  quality  and 
many  new  operating  modes. 

At  the  simplest  level,  the  DMS  provides  the  aircrew  with  a  display  of  standard  paper  aeronautical  charts 
that  have  been  digitized  and  stored  inside  the  DMS,  as  seen  in  Figure  1.  The  aircrew  can  select  from  several 
scales  of  aeronautical  charts  and,  since  the  DMS  receives  information  from  the  aircraft  navigation  equipment,  it 
can  move  the  map  along  with  the  aircraft  to  always  show  the  area  around  the  aircraft's  location.  The  map  can  be 
displayed  north  up  or  track  up  or  heading  up  Utilizing  terrain  elevation  data,  the  DMS  can  also  display  terrain 


Figurel  Figure  2 

Example  Aeronautical  Chart  Example  Terrain  Elevation  Display 


relief  maps  with  contour  lines  at  user  specified  intervals,  as  seen  in  Figure  2.  The  DMS  can  also  create  moving 
displays  of  satellite  photo  data,  as  seen  in  Figure  3.  This  data  is  currently  available  in  5  and  1  0  meter  resolution. 
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All  three  of  these  DMS  modes  are  created  from 
standard  database  products  provided  by  the  National 
Imagery  and  Mapping  Agency  (NIMA).  Aeronautical 
charts  are  created  from  Compressed  Arc  Digitized  Raster 
Graphics  (CADRG),  terrain  relief  is  created  from  Digital 
Terrain  Elevation  Data  (DTED)  and  satellite  photo  maps 
are  generated  from  Controlled  Image  Base  (C1B).  These 
standard  database  products  are  the  same  ones  used  on 
aircraft  mission  planning  systems,  which  ensures  that  the 
maps  displayed  in  the  cockpit  are  identical  to  those  seen 
during  mission  planning.  Also,  since  the  DMS  reads 
database  files  exactly  as  they  come  from  NIMA,  database 
management  is  simplified  and  load  times  are  significantly 
improved  when  compared  with  earlier  digital  map 
products. 
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Figure  3 

Example  Satellite  Photo  Display 


The  DMS  provides  several  scales  of  map  imagery'  as  show  n  in  the  table  below: 


Scale  Set 

Scale  Set 

Aeronautical  Chart 

Satellite  Photo 

Terrain  Elevation 

1  1 

2 

(CAI)RG) 

(CIB) 

(DTED) 

160  nmi 

GNC  (1:5  M) 

80  nmi 

104.22  nmi 

JNC  (1:2  M) 

Level  1 

40  nmi 

52.1 1  nmi 

ONC  (1:1  M) 

Level  1 

20  nmi 

26.05  nmi 

TPC  (1:500  K) 

Level  1 

10  nmi 

13.03  nmi 

JOG  (1:250  K) 

Level  1 

10  nmi 

5.21  nmi 

TLM  (1:100  K) 

Level  1 

2  nmi 

2.61  nmi 

TLM(I:50  K) 

1 0  meter 

1  nmi 

1.305  nmi 

5  meter 

0.5  nmi 

City  Graphics  (1:12.5  K) 

The  three  modes  discussed  so  far  are  all  geo¬ 
graphically  referenced,  that  is,  they  represent  specific 
areas  of  the  earth  and  the  DMS  knows  the  exact  latitude 
and  longitude  of  every  point  in  the  image.  The  DMS  can 
also  display  non-  geographically  referenced  Data  Frames, 
as  shown  in  Figure  4.  Data  Frames  are  created  by 
scanning  any  type  of  hard  copy  information  and  storing  it 
digitally.  This  would  include  check  lists,  diagrams, 
approach  plates,  reconnaissance  images,  or  anything  else 
that  can  be  carried  to  the  scanner  at  a  mission  planning 
station  Data  Frames  can  be  rotated,  panned,  and  zoomed. 

Overlays 

Once  the  background  layer  of  the  digital  map  has  been 
established,  several  types  of  overlay  graphics  can  be  applied  Figure  4 

that  are  pertinent  to  situational  awareness.  Figure  5  shows  Example  Data  Frame 

several  types  of  overlay  information  applied  to  a  Terrain  Elevation  Display.  Letter  designations  in  the  figure  pertain  to 
the  references  in  the  following  text.  For  purposes  of  brevity,  not  all  overlays  are  depicted  in  this  paper.  In  general, 
overlays  can  be  segregated  into  two  classes,  geographic  oriented  and  screen  oriented 
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The  table  below  indicates  several  types  of  screen  oriented  overlays  that  the  baseline  TAMMAC  DMS  supports. 

Example  Screen  Oriented  Overlays 


General 

MFD  Soft  Key  Legends 

(A) 

^  Caution  and  Alert 
Messages 

Stores  Status 


Map  Control/Status 

^  For  Chart,  Photo,  &  Elevation: 

•  Scale  Displayed  (B) 

•  Zoom  Factor  Applied 
^  For  Elevation: 

•  Contour  Line  Interval  (C) 
For  Data  Frames: 

•  Data  Frame  Number 


Navigational 

^  Aircraft  or  Cursor  Position 

•  In  Latitude/Longitude 

•  InMGRS 

•  InUTM 

4*  Air  or  Ground  Speed  (D) 

*$■  Heading  Angle  (E) 

4-  Elevation 

Distance  and  Time  to  Go  to 
Next  Waypoint  (F) 

*$■  Range  and  Bearing  to 
Cursor  (G) 

4*  Time  on  Target 

^  Compass  Rose/Arrow 

^  HSI 


Figure  5 

Overlay  Information  Applied  to  a  Terrain  Elevation  Map 
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Complex  screen  oriented  graphics  can  be  created  on  the  TAMMAC  DMS  by  employing  its  high  order 
macro  language.  Examples  of  F/A- 1 8  nominal  and  worst  case  overlay  graphics  are  shown  below. 


The  DMS  can  create  these  screens  either  overlaid  onto  a  digital  map  background  or  with  the  map  blanked  as 
shown  here. 

The  second  type  of  overlays  generated  by  the  DMS,  geographic  oriented,  are  features  associated  with  the  terrain. 
Geographic  overlays  can  be  applied  to  aeronautical  charts,  terrain  elevation,  and  satellite  photo  displays.  Perhaps  the 
most  common  examples  of  geographic  overlays  are  the  aircraft  symbol  (J),  waypoints  (K),  and  flight  plans  (L).  The 
DMS  can  display  primary  and  alternate  flight  plans  generated  during  mission  planning  and  fully  supports  in  flight  route 
replanning.  A  Hook  Cursor  is  provided  to  allow  lift,  drag,  and  drop  of  individual  waypoints.  Any  number  of  flight 
plans  can  be  managed  by  the  mission  computer  and  displayed  on  the  DMS.  Closely  associated  with  the  flight  plan  are 
Trend  Dots.  Trend  Dots  are  three  dots  that  appear  in  front  of  the  aircraft  symbol  that  indicate  the  predicted  location  in 
10,  20,  and  30  seconds  respectively1.  They  respond  to  changes  in  velocity  and  turn  rate  and  move  on  screen  as  the 
aircraft  maneuvers.  An  example  can  be  seen  in  Figure  5.  Pilot  feedback  regarding  Trend  Dots  from  MH-53J  flight 
testing  has  been  very  favorable.  One  case  cited  was  that  when  flying  a  blind  turn  through  a  valley,  the  pilot  could  adjust 
the  aircraft  turn  to  place  the  Trend  Dots  in  the  center  of  the  valley  and  the  aircraft  responded  exactly  as  the  Trend  Dots 
predicted 

There  are  actually  two  types  of  geographic  overlays  provided  by  the  DMS,  symbology  and  tints.  Symbology 
overlays  include  lines,  circles,  polygons,  symbols,  and  text.  Tints  apply  a  translucent  color  over  the  background.  A 
tinted  overlay  is  used  to  indicate  areas  on  the  map  where  the  terrain  is  above  the  aircraft  altitude  (M).  This  capability  is 
sometimes  referred  to  as  Height  Above  Terrain  (HAT)  and  sometimes  Elevation  Color  Banding  although  neither  term 
is  particularly  accurate  and  the  latter  has  another  meaning  also.  For  the  purposes  of  this  paper,  the  capability  will  be 
referred  to  as  Terrain  Above  Aircraft  or  TAA.  The  DMS  constantly  processes  the  terrain  elevation  database  in  its  mass 
memory  in  order  to  determine  the  areas  to  tint.  One  color  is  used  to  indicate  the  areas  that  are  above  the  current  aircraft 
altitude  and  a  second  tint  color  is  used  to  show  the  areas  that  are  below  the  aircraft  altitude  but  above  the  set  clearance 
altitude.  The  tinted  areas  change  size  as  the  aircraft  changes  altitude  to  always  convey  to  the  flight  crew  an  indication 
of  safe  flying  areas.  Since  the  tint  applied  over  the  map  is  translucent,  all  details  from  the  background  map  always 
remain  readable  through  the  tint. 

The  DMS  also  provides  a  Clear  Line  of  Sight  (CLOS)  indication  either  from  the  aircraft  location  to  a  point  or 
between  any  two  points.  A  line  is  shown  on  the  display  to  show'  CLOS  with  the  line  being  broken  by  any  intervening 
terrain.  The  location  of  the  obstruction  is  provided  by  the  DMS  for  use  by  other  systems. 


1  The  Trend  Dot  intervals  are  actually  programmable  values  that  can  be  configured  uniquely  by  the  user  for 
different  types  of  aircraft.  Any  practical  time  interval  can  be  specified 
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Threat  Processing 

The  TAMMAC  DMS  aLo  provides  a  new  level  of  capability  tr  display  details  of  the  threat  environment  sur¬ 
rounding  the  aircraft.  First,  threat  locations  are  denoted  by  the  presence  of  threat  symbols  on  the  map.  Each  unique 
threat  type  is  represented  by  a  particular  threat  symbol  (N).  The  list  of  threat  symbols  is  loaded  onto  the  DMS  along 
with  the  map  database  information  and  can  be  unique  for  each  platform  type.  Each  threat  can  have  an  associated 
legend  displayed.  Threats  can  also  have  an  associated  threat  ring  shown  that  identifies  the  range  of  its  lethality  (O). 
Like  way^ints,  threats  can  be  lifted,  dragged,  and  dropped  by  the  *  »ook  Cursor. 

A  new  dimension  to  situational  awareness  is  provided  by  the  threat  intervisibility  overlay.  Threat  intervisibility  is 
another  tinted  overlay  that  shows  the  areas  where  the  aircraft  can  be  seen  from  a  threat,  given  its  altitude  above  ground 
level  relative  to  the  surrounding  terrain  (P).  The  DMS  continually  processes  the  terrain  elevation  database  to  calculate 
line  of  sight  visibility  between  the  threat  site  and  the  aircraft,  with  antenna  height  of  the  threat  taken  into  account.  The 
threat  intervisibility  overlay  is  continuously  updated  as  the  aircraft  changes  altitude  so  that  tinted  areas  always  show 
where  the  aircraft  would  be  detected.  Conversely,  untinted  areas  depict  areas  where  the  aircraft  would  remain  hidden. 
Like  TAA,  the  tint  applied  is  translucent  so  that  the  nothing  on  the  background  map  is  obscured.  The  range  to  display 
threat  intervisibility  can  be  different  for  each  threat  type  in  order  to  accurately  reflect  its  detection  range.  Also,  a  threat 
penetration  indication  can  be  enabled  to  alert  the  flight  crew  when  the  aircraft  has  entered  a  threat’s  range. 

Threats  are  organized  into  several  categories  with  each  category  occupying  a  separate  layer.  The  flight  crew  can 
choose  to  display  all  threat  categories  or  can  declutter  the  display  by  only  choosing  certain  categories  for  display.  It  is 
expected  that  initially  most  of  the  threats  depicted  on  the  map  will  be  preplanned,  that  is,  threats  that  were  known  at  the 
time  of  mission  planning  and  transferred  to  the  aircraft  along  with  other  mission  data.  However,  the  DMS  also  supports 
the  display  of  pop  up  threats,  which  are  any  new  threats  that  are  identified  during  flight.  Pop  up  threats  can  be 
identified  by  either  on-board  or  off-board  sensors.  Pop  up  threats  can  be  configured  to  appear  different  from  pre¬ 
planned  threats  Typically,  pop  up  threats  would  have  the  same  symbol  shapes  as  they  would  if  they  had  been 
preplanned  but  would  have  different  symbol  colors  or  attributes  (R). 

In  addition  to  the  specific  types  of  geographic  overlay  features  described  above,  the  DMS  provides  a  generic  in¬ 
terface  to  allow  any  type  of  geographically  referenced  information  to  be  displayed.  Several  independent  geographic 
layers  are  provided  that  can  each  contain  symbols,  lines,  circles,  polygons,  and  text  strings.  These  layers  can  be 
managed  to  show  information  such  as  targets,  ground  force  locations,  restricted  airspace,  downed  airman  indications, 
CHUM  data,  or  any  other  type  of  entry  .  Each  layer  can  be  selectively  enabled  to  allow  the  flight  crew  to  declutter  the 
display. 

Image  Positioning 

The  TAMMAC  DMS  provides  considerable  flexibility  in  controlling  the  map  image.  When  displaying  aeronau¬ 
tical  charts,  satellite  photo,  or  terrain  elevation  displays,  the  image  can  be  oriented  north  up  or  alternatively  in  a  track  up 
or  heading  up  orientation.  For  helicopter  applications  the  map  can  be  controlled  to  automatically  orient  heading  up 
when  hovering  and  at  low  speed  and  then  switch  to  a  track  up  orientation  as  speed  increases.  Aircraft  position  on 
screen  can  be  in  the  center  of  the  display  or  offset  to  show  more  area  ahead  of  the  aircraft.  This  decenter  position  can 
be  defined  differently  for  each  platform.  When  desired,  the  map  can  be  commanded  to  display  a  location  other  than  at 
the  aircraft  location,  such  as  a  target  area,  landing  zone,  or  future  waypoint.  When  the  map  is  displaying  imagery  that  is 
not  centered  on  the  aircraft,  an  alternate  center  of  interest  symbol  can  be  enabled  to  denote  the  focal  point  selected.  All 
display  modes  of  the  DMS  can  be  rotated,  panned,  and  zoomed.  Zooming  can  either  be  in  discrete  steps,  or  be  smooth 
and  continuous.  Displays  can  be  zoomed  out  as  far  as  0.5: 1  and  zoomed  in  as  far  as  8: 1 . 

All  of  the  capabilities  of  the  DMS  are  available  on  two  completely  independent  channels.  As  an  example,  one 
channel  can  be  viewed  by  the  pilot  at  the  aircraft  location  in  track  up  orientation  with  a  set  of  overlays  he  has  selected 
while  the  other  channel  can  support  a  weapons  officer  who  is  viewing  the  target  area  in  north  up  orientation  with  a 
different  set  of  overlays  applied.  Each  channel  provides  both  a  color  and  a  monochrome  output  to  facilitate  the  displays 
particular  to  any  aircraft. 

Because  the  aircraft  slated  to  use  the  TAMMAC  DMS  vary  so  much  in  aircraft  performance,  the  DMS  was  de¬ 
signed  to  support  a  particularly  wide  range  of  aircraft  dynamics.  It  provides  smooth  image  motion,  regardless  of  scale, 
at  any  speed  or  turn  rate.  It  can  change  scale  or  jump  to  any  position  in  the  world  in  less  than  one  second. 

Flexibility 

The  TAMMAC  DMS  was  designed  from  the  start  to  support  a  wide  variety  of  aircraft.  Platforms  ranging  from 
utility  and  attack  helicopters  to  fighter  and  attack  jets  provided  the  initial  requirements  that  shaped  the  DMS  specifica¬ 
tion.  This  direct  involvement  from  so  many  types  of  aircraft  has  ensured  that  the  DMS  will  provide  the  capabilities 
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needed  to  execute  a  diverse  range  of  missions.  Although  providing  these  capabilities  goes  a  long  way  toward  support¬ 
ing  numerous  kinds  of  aircraft,  there  will  always  be  individual  platform  preferences  about  the  exact  appearance  of  the 
digital  map  backgrounds  and  overlays. 

In  order  to  allow  the  appearance  of  the  digital  map  displays  to  be  tailored  to  satisfy  these  preferences,  the 
TAMMAC  DMS  utilizes  a  Configurable  Parameters  Table.  The  table  controls  the  colors  and  attributes  associated  with 
features  such  as  waypoints,  flight  plans,  threats,  pop  up  threats,  cursor,  threat  intervisibility  tint  and  TAA  tint,  rhe  table 
also  contains  timing  and  voltage  level  definitions  for  the  video  outputs.  Color  remap;  mg  tables  are  provided  to  allow 
special  color  palettes  and  NV1S  compatible  colors  to  be  used.  Each  platform  can  have  its  own  symbol  set.  Symbol 
shapes  can  be  created  by  the  user  with  standard  PC  tools.  Once  created,  these  symbols  are  combined  into  a  symbol 
table  that  is  downloaded  to  the  DMS.  The  symbol  set,  color  tables,  and  configurable  parameters  table  can  all  be  loaded 
into  the  DMS  without  removing  it  from  the  aircraft. 

Despite  the  high  degree  of  flexibility  provided  by  the  TAMMAC  DMS,  it  is  expected  that  as  additional  platforms 
consider  it  for  use,  new  modes  and  capabilities  will  be  identified.  Also,  evolution  in  the  baseline  aircraft  will  drive  the 
need  to  expand  the  DMS’s  capabilities.  For  these  reasons  the  TAMMAC  DMS  was  designed  to  change  and  evolve 
without  becoming  obsolete.  The  initial  set  of  capabilities  of  the  DMS  utilizes  less  than  half  of  its  available  capacity.  In 
addition,  not  only  can  its  software  be  reprogrammed  but,  through  the  use  of  Field  Programmable  Gate  Array  integrated 
circuits,  the  DMS  can  actually  change  its  circuit  design.  All  of  the  DMS  software,  circuit  configuration,  and  configur¬ 
able  parameters  can  be  changed  in  the  field  through  simple  operations. 

Data  Loading 

Data  is  loaded  into  the  DMS  via  the  TAMMAC  AMU  and  is  performed  in  two  steps.  First,  when  the  unit  is  de¬ 
ployed  to  a  new  area,  a  theatre  load  is  performed.  The  theatre  load  would  normally  be  accomplished  by  the  mainte¬ 
nance  crew  chief  and  would  consist  of  all  the  paper  chart,  terrain  elevation,  and  satellite  photo  data  available  for  the 
region  along  with  any  general  Data  Frames  that  would  be  utilized  repeatedly  during  missions.  Sufficient  mass  storage 
is  provided  by  the  DMS  to  guarantee  full  area  coverage  over  entire  mission  areas.  Map  database  files  are  loaded  onto 
Type  3  PCMCIA2  cards  at  the  mission  planning  station,  transported  to  the  aircraft,  and  inserted  into  the  AMU  from 
which  the  data  is  downloaded  to  the  DMS.  A  full  theatre  download  of  3  Gbytes  of  database  files  can  be  accomplished 
in  approximately  30  minutes.  Since  loading  is  performed  through  the  AMU,  the  DMS  can  be  located  deep  within  the 
aircraft  in  order  to  avoid  use  of  precious  cockpit  space.  Once  the  theatre  area  is  loaded,  the  DMS  is  ready  to  fly  and  the 
theatre  load  cards  are  no  longer  needed. 

The  second  step  in  loading  data  into  the  DMS  is  to  install  mission  specific  data.  Mission  data  would 
include  information  such  as  flight  plans,  the  threat  environment,  any  restricted  airspace,  reconnaissance  photos, 
CHUM,  and  any  updates  to  theatre  data  files  -  in  short,  any  data  available  on  the  mission  planning  station. 
Mission  data  is  loaded  onto  flightworthy  Type  2  PCMCIA  cards  by  the  flight  crew  at  the  mission  planning 
station,  transported  to  the  aircraft,  and  inserted  into  the  AMU. 

Growth 

Although  the  digital  map  capabilities  described  so  far  provide  a  significant  improvement  in  situational  awareness 
for  today’s  flight  crews,  many  more  enhancements  will  be  added  shortly  that  will  continue  to  improve  the  TAMMAC 
DMS  for  the  flight  crews  of  tomorrow.  The  DMS  provides  its  current  capabilities  while  only  using  half  of  its  available 
resources.  That  leaves  a  considerable  capacity  to  support  the  addition  of  new  features. 

Databases 

Some  of  the  initial  enhancement  that  will  occur  will  be  in  the  type  of  the  database  products  that  the  DMS  can  dis¬ 
play.  More  scales  of  City  Graphics  paper  chart  data,  along  with  higher  resolution  C1B  and  DTED  will  be  available 
shortly.  Also,  NIMA  is  developing  a  new  type  of  database  known  as  Vector  Product  Format  (VPF).  This  type  of  data 
keeps  track  of  all  of  the  cultural  feature  information  that  is  contained  on  a  map  Use  of  VPF  data  will  allow  text  labels 
and  legends  to  always  appear  upright  on  the  map  display  regardless  of  how  the  map  is  rotated.  Also,  cultural  features 
will  be  able  to  be  decluttered  by  category  so  that  only  the  desired  information  is  displayed. 


2  PCMCIA  stands  for  Personal  Computer  Memory  Card  and  Industry'  Association.  The  cards  are  used  widely  in 
laptop  PCs  for  a  variety  of  functions  such  as  modems,  special  interfaces,  and  memory.  PCMCIA  cards  are 
available  in  Type  I,  2,  &  3  with  each  type  having  a  different  thickness.  Flash  EEPROM  is  typically  available  in 
Type  2  cards  with  densities  ranging  from  85  Mbytes,  now,  up  to  350  Mbytes  in  the  next  few  years.  Type  3  cards 
can  contain  a  miniature  hard  disk  drive  which  allows  significantly  higher  memory  capacity. 
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Other  database  types  under  consideration  for  use  by  the  DMS  include  digital  FLIP  charts  (DFL1P),  Digital 
Bathymetric  Database  (DBDB),  Controlled  Multispectral  Image  Base  (CMIB),  Digital  Topographic  Data  (DTOP), 
Digital  Nautical  Chart  (DNC),  arid  Digital  Point  Position  Database  (DPPDB).  Of  course  the  addition  of  all  these  new 
database  types,  along  with  DTED  and  CIB  data  down  to  2  and  I  meter  resolutions  greatly  increases  the  amount  of  mass 
memory  needed  in  the  DMS  to  store  it  all.  That  is  why  the  DMS  mass  storage  is  implemented  in  a  ruggedized  hard 
disk  drive.  As  the  laptop  personal  computer  market  continues  to  push  hard  drive  capacities  higher  and  their  costs  lower, 
these  newer  hard  driven  can  be  dropped  into  the  ruggedized  disk  housing  inside  the  DMS  to  increase  its  storage 
capacity. 

Ground  Collision  Avoidance  Warning 

Just  as  the  stored  terrain  elevation  data  is  used  to  deter¬ 
mine  what  terrain  is  above  the  aircraft,  it  can  also  be  used  to 
predict  when  the  aircraft  is  headed  for  a  collision  with  the 
ground.  Techniques  of  this  sort,  for  preventing  Controlled 
Flight  into  Terrain  (CF1T),  are  becoming  knowii  as  Predictive 
Ground  Proximity  Warning  Systems  (PGPWS).  Existing 
GPWS  equipment  considers  only  the  terrain  directly  under  the 
aircraft  for  predicting  CFIT  situations.  The  performance  of 
these  systems  has  been  acceptable  over  flat  terrain  but  has  not 
been  particularly  effective  in  mountainous  areas1.  As  a  result, 
the  Navy  has  begun  an  effort  to  enhance  the  performance  of 
existing  GPWS  algorithms  by  using  stored  terrain  data.  It  is 
envisioned  to  have  the  new  PGPWS  algorithms  operate  in  the  Figure  6 

TAMMAC  DMS  Using  the  stored  terrain  database  in  this  PGPWS  Display 

fashion  is  considered  by  many  to  be  the  first  step  toward  employing  the  terrain  data  for  passive  TFATA. 

Enhanced  Threat/T arget  Processing 

Another  growth  area  for  the  TAMMAC  DMS  is  in  the  processing  and  display  of  the  threat  environment.  The  ex¬ 
isting  capability  allows  the  display  of  a  static  threat  ring  indicating  the  lethality  range  for  threats.  This  capability  can  be 
enhanced  to  utilize  aircraft  Radar  Cross  Section  (RCS)  along  with  its  orientation  and  velocity  relative  to  the  threat  to 
provide  a  dynamic  threat  ring  that  changes  shape  and  dimension  as  the  aircraft  maneuvers. 

A  capability  that  is  already  in  use  on  a  digital  map  in  the  MH-53J  helicopter  is  integration  of  the  DMS  with  off- 
board  sensors  to  provide  a  near  real  time  update  of  the  Electronic  Order  of  Battle  (EOB).  In  that  system  the  DMS 
receives  new  pop-up  threat  information  from  a  secure  satcom  radio.  Transfer  of  offboard  sensor  data  to  an  aircraft  for 
display  on  the  digital  map  is  now  being  referred  to  as  Real  Time  Information  in  the  Cockpit  or  RT1C.  RTIC  data 
transfers  are  not  limited  to  new  threat  information.  Flight  demonstrations  have  been  conducted,  using  a  digital  map  that 
preceded  TAMMAC,  where  near  real  time  imagery  was  been  sent  to  the  aircraft  for  display  on  the  digital  map  as  an 
inset  “picture-in-picture”  window.  A  follow  on  demonstration  will  expand  the  capability  by  passing  new  target 
information  to  the  DMS. 

Operation  with  Other  IS  ew  Aircraft  Equipment 

Much  of  the  growth  areas  expected  for  the  TAMMAC  DNflS  are  in  the  area  of  stay  ing  up  to  date  as  other  new 
equipment  are  added  to  the  aircraft.  New  displays  will  have  larger  screens  and  call  for  higher  resolution  imagery'. 
Many  of  these  displays  will  no  longer  accept  the  traditional  analog  video  interface  used  today.  Digital  video  outputs 
will  be  required.  Also,  some  users  will  have  a  need  for  a  third  independent  video  output.  Also,  new  interfaces,  such  as 
Fibre  Channel,  are  being  considered  to  replace  M1L-STD-1553.  All  of  these  areas  are  included  in  the  TAMMAC  DMS 
specification  as  growth  areas  that  the  equipment  must  be  able  to  support. 

Perspective  View 

Another  enhancement  that  is  included  in  the  TAMMAC  DMS  road  map  is  the  addition  of  a  3-D  “out  the  win¬ 
dow”  perspective  view.  Processing  of  the  map  database  would  be  performed  to  render  an  image  that  would  approxi¬ 
mate  the  view  seen  from  the  aircraft  window.  A  display  of  this  type  could  be  used  for  correlation  with  onboard  FLIR 
sensors  as  the  aircraft  flies  or,  alternatively,  could  provide  an  in  cockpit  mission  rehearsal  capability. 

Conclusion 

The  TAMMAC  DMS  will  provide  an  unprecedented  array  of  new  capabilities  for  aircrews  to  enhance  situational 
awareness  and  improve  mission  effectiveness.  The  baseline  capabilities  of  the  system  will,  in  themselves,  allow 
platforms  to  tailor  the  DMS  operation  to  suit  the  needs  of  their  particular  missions.  Considerable  thought  has  already 


been  given  to  identifying  and  shaping  the  growth  capabilities  described  for  the  TAMMAC  DMS.  However,  a 
methodical  study,  including  flight  crew  interaction  and  human  factors  activities,  should  be  undertaken  to  guide  the 
platforms  in  deciding  how  to  manage  the  considerable  growth  capabilities  offered  by  the  system. 


References 

1 .  Digital  Map  Requirements  Study  in  Support  of  Advanced  Cockpit  Moving  Map  Displays”;  M.  Lohrenz,  et  al, 
1997. 

2.  “A  Human  Factors  Engineering  Analysis  of  Digital  Moving  Map  Systems  for  the  USMC  AH-1 W  Superco¬ 
bra”;  J  Ruffner  and  P.  Puccetti,  1996. 

3.  “F/A-18C/D  Hornet  Ground  Proximity  Warning  System  Design  Report”  8  October  1997. 


112 


Promoting  Situational  Awareness  with  the  Tammac  Digital  Map 
System:  Human  Factors  Research  and  Design  Issues 


John  W.  Ruffner 

Night  Vision  and  Avionics  Department 
DCS  Corporation 
Alexandria,  VA 


Michael  E.  Trenchard 
Naval  Research  Laboratory 
Mapping  Sciences  Section 
Stennis  Space  Center,  MS 


Abstract 

The  US  Navy's  Tactical  Aircraft  Moving  Map  Capability  (TAMMAC)  program  will  provide  the 
standard  cockpit  digital  moving  map  system  for  Naval  aviation.  The  TAMMAC  system  will  be  used  by  a 
variety  of  Navy  aircraft  with  differing  operational  needs  and  resources  and  can  be  tailored  to  meet  each 
aircraft’s  operational  requirements.  A  major  design  goal  of  the  TAMMAC  program  is  to  enhance  situational 
awareness  (SA)  and  aircrew  mission  effectiveness  without  further  burdening  pilot  workload.  The  TAMMAC 
system  has  been  designed  to  give  pilots  a  great  deal  of  flexibility'  to  tailor  the  map  capabilities  to  meet 
platform-specific  needs.  There  is  a  need  to  structure  the  TAMMAC  user  interface  to  take  maximum  advantage 
of  these  capabilities  to  promote  SA  without  contributing  to  pilot  workload.  The  existing  TAMMAC  require¬ 
ments  and  capabilities  are  largely  based  on  user  preferences  from  limited  demonstrations  of  preliminary 
moving-map  capabilities.  Therefore,  there  is  a  need  to  test  the  applicability  of  these  capabilities  for  promoting 
SA  using  part-task  trainers  or  simulators.  The  paper  summarizes  human  factors  research  relevant  to  SA  with 
digital  moving  map  systems  and  identifies  research  and  design  needs  for  promoting  SA  with  the  TAMMAC 
system. 

Introduction 

Background 

The  US  Navy  is  currently  sponsoring  a  Tactical  Aircraft  Moving  Map  Capability  (TAMMAC)  program  that 
will  provide  the  standard  cockpit  digital  moving  map  system  for  Naval  Aviation.  The  TAMMAC  system  consists  of  a 
Digital  Map  System  (DMS),  an  Advanced  Memory  Unit  (AMU)  for  loading  mission  planning  data  and  logging 
maintenance  data,  and  a  High  Speed  Interface  Bus.  TAMMAC  will  be  used  by  a  variety  of  Navy  aircraft  with 
different  operational  needs.  It  can  be  tailored  to  meet  each  aircraft’s  operational  requirements  by  selecting  from 
several  capabilities.  Many  of  these  capabilities  are  discussed  in  a  separate  paper  presented  at  this  symposium 
(Williams,  1998) 

A  major  design  goal  of  the  TAMMAC  program  is  to  enhance  SA  and  aircrew  mission  effectiveness  without 
further  burdening  pilot  workload  (Lohrenz,  Trenchard,  Myrick,  Van  Zuyle,  Pemiciaro,  Gendron,  and  Brown,  1977a; 
Ruffner  and  Trenchard,  1997).  Since  every  cockpit  display  and  control  is  accompanied  by  some  additional 
workload  burden,  it  is  important  that  new  capabilities,  such  as  a  digital  map  system,  be  integrated  into  the  aircraft  so 
that  informational  elements  are  organized  for  the  pilot's  most  direct  comprehension  and  subsequent  application 
(Rogers  and  Spiker,  1 988). 

Organization  of  the  Paper 

The  paper  is  organized  into  six  sections.  In  the  first  section,  we  briefly  review  the  construct  of  S A  as  a  pre¬ 
cursor  to  the  following  discussion  of  the  role  of  SA  in  the  design  and  utilization  of  digital  moving  map  systems.  In 
the  second  section  we  review  selected  studies  relevant  to  SA  in  digital  moving  map  systems.  We  then  briefly  discuss 
the  primary  TAMMAC  baseline  and  growth  features.  In  the  fourth  section,  we  discuss  SA-design  guidelines. 
Following  this,  we  identify  human  factors  research  and  design  issues  for  SA  in  digital  moving  map  systems.  In  the 
final  section,  we  present  our  conclusions  and  recommendations  for  promoting  SA  with  the  TAMMAC  Digital  Map 
System  (DMS). 
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Situational  Awareness 


Situational  awareness  (SA)  is  a  construct  that  has  been  applied  to  a  variety  of  aviation  tasks  and  settings  and  is  usually 
considered  important  for  mission  effectiveness  and  safety.  Even  though  it  is  not  currently  possible  to  specify  how 
much  SA  is  enough  for  a  particular  aviation  mission  or  task,  it  is  eommonly  believed  that  good  performance  is  linked 
to  good  SA  (Endsley,  1995). 

Although  many  definitions  for  S  A  have  been  offered,  they  have  several  elements  in  eommon.  For  example, 
Wickens  ( 1 995,  1996)  defined  SA  as  tl  .  continuous  extraction  of  information  about  a  system  or  environment,  the 
integration  of  this  information  with  previous  knowledge  to  form  a  coherent  mental  picture,  and  the  use  of  that  picture 
in  directing  further  perception,  anticipation,  attention  or  responding.  This  definition  closely  parallels  that  proposed  by 
Endsley  (1995)  in  which  SA  is  defined  as  the  perception  of  the  critical  elements  of  an  environment  in  time  and  space 
(Level  1).  the  comprehension  of  their  meaning,  particularly  when  integrated  in  relation  the  airerew’s  goals  (Level  2), 
and  the  projection  of  what  will  happen  with  the  system  in  the  near  future.  It  is  widely  believed  that  the  higher  levels 
of  SA  allow  pilots  to  function  in  a  timely  and  effective  manner. 

In  addition,  SA  can  be  global  or  local  (Endsley,  1997a).  Global  SA  information  needs  relevant  to  digital 
moving  map  systems  include  one’s  location  within  a  broad  geographical  area,  navigation  information  such  as  the 
relative  location  of  important  features,  the  current  location  and  direction  of  movement  of  friendly  and  enemy  units, 
and  current  commands  and  directions.  All  these  factors  are  relevant  to  the  aircrew’s  ability  to  navigate  and  plan 
strategically  to  meet  their  goals.  Local  SA  needs  include  the  location  of  a  desired  target  in  the  immediate  environ¬ 
ment,  the  identity  (friend,  foe,  or  neutral)  of  an  entity'  under  current  targeting,  terrain  and  object  location  (as  needed 
for  maneuvering),  and  eueing  of  the  presence  and  movement  of  threats  in  the  immediate  environment.  This  informa¬ 
tion  is  eritieal  to  the  aircrew’s  basic  awareness  of  threat  presence  in  relation  to  the  aircraft  and  the  ability  to  reaet 
quiekly  in  accordance  with  mission  goals.  Both  global  and  local  SA  needs  are  eritieal  for  effective  aircrew  func¬ 
tioning  in  a  given  environment. 

Reeent  research  suggests  that  SA  is  most  likely  a  multidimensional  construct.  For  example,  Endsley 
( 1 097a)  proposed  that  several  classes  of  elements  were  required  for  SA:  geographical,  spatial/temporal,  system, 
environmental,  and  taetieal  Wiekens  (1995)  suggested  that  overall  SA  should  be  broken  down  into  hazard  aware¬ 
ness,  system  awareness,  and  task  awareness.  Furthermore,  Coury  and  Wilson  (1994)  identified  five  SA  aspects: 
spatial,  identity,  temporal,  responsibility,  and  expectancy.  Some  of  these  classes  of  SA  elements  (e.g.,  geographical, 
tactical)  will  be  more  relevant  to  promoting  SA  in  digital  moving  map  systems  than  others  (e.g.,  system,  environ¬ 
mental). 

Research  Relevant  to  SA  in  Digital  Moving  Map  Systems 

Digital  moving  map  systems  are  designed  to  replaee  cumbersome  paper  maps  in  aireraft  cockpits.  They 
provide  information  useful  for  navigation  and  taetieal  tasks,  and  ean  provide  a  means  for  enhancing  SA.  They 
allow  the  pilot  to  foeus  his  or  her  attention  on  navigation  with  a  minimum  amount  of  head-down  time  Digital 
moving  map  systems  integrate  information  from  several  sources.  When  properly  designed,  they  can  serve  to 
display  information  the  pilot  needs  more  efficiently  such  that  the  pilot  should  be  able  to  obtain  all  the  informa¬ 
tion  needed  to  assess  a  situation  and  aeeomplish  a  task  with  a  quick  glance  at  the  display  (Unger  and  Sehopper, 
1995).  In  addition  digital  moving  map  systems  provide  the  aircrew  with  control  of  the  displayed  information 
A  number  of  studies  have  been  conducted  during  the  last  20  years  to  identify  the  functional  requirements  and 
desirable  features  and  capabilities  for  digital  moving  map  systems.  In  this  seetion,  we  briefly  summarize 
selected  digital  moving  map  system  studies  relevant  to  SA 

U.S.  Army  Digital  Map  Functional  Requirements  Analysis 

A  comprehensive  program  to  identify  the  functional  requirements  of  ground  based  mission  planning 
systems  and  airborne  digital  map  systems  for  US  Army  aviators  was  conducted  during  the  1980’s.  This 
program  drew  on  the  findings  of  investigations  on  helieopter  navigational  requirements,  map  usage,  and  the 
effects  of  various  display  variables  on  the  perception  of  topographic  features  and  symbology  (e.g.,  Rogers, 
1983;  Rogers  and  Cross,  1979;  Rogers,  Gutmann,  and  Ralstin,  1982). 

During  both  mission  planning  and  flight,  the  aviator  must  extract  a  great  deal  of  information  from 
maps.  Rogers  (1985)  notes  that  the  aviator  must  study  and  visualize  the  overall  situation  and  topography; 
select  engagement  points,  observation  points,  or  landing  zones;  determine  primary  and  alternate  (masked) 
routes  of  flight;  select  air  control  points  (ACPs),  checkpoints,  and  barrier  features;  and  determine  flight 
modes,  altitudes,  speeds,  and  durations.  Each  of  .hese  activities  places  a  large  burden  on  the  aviator’s 


114 


information  compilation  and  processing  skills. 

Early  map  displays  were  designed  for  assisting  in  navigation;  however  the  greatest  contributions  of  a 
cockpit  digital  moving  map  display  are  likely  to  be  in  aiding  the  performance  of  both  mission  planning  and 
tactical  decision-making  tasks.  Rogers  (1985)  identified  four  potential  advantages  of  a  computer-generated 
topographic  display  system  in  addition  to  enhancing  navigation  capability: 

1 .  Potential  for  comprehensive  and  rapid  response  and  cartographic  support.  In  contrast  to  the  long 
lead  time  required  for  conventional  or  photo-based  maps,  it  is  possible  to  obtain  the  data  required  to 
support  computer-generated  display  systems  within  hours  vice  weeks  or  months. 

2.  Control  of  the  content  of  the  displayed  information.  Because  of  the  variety  of  roles  that  he  or  she  is 
expected  to  fill,  the  aviator  may  need  different  types  of  information  on  different  missions  or  in  differ¬ 
ent  phases  of  flight  Furthermore,  map  clutter  must  be  avoided.  Aviators  using  a  computer-generated 
map  can  select  the  information  that  is  optimal  for  the  situation  at  hand,  can  control  the  classes  of  in¬ 
formation  that  are  displayed  (e.g.,  vegetation,  hydrography),  and  can  select  the  specific  features  of  a 
given  class  of  information  (e  g.,  deciduous  trees,  perennial  streams).  In  addition  the  scale  and  contour 
interval  can  be  altered  to  tailor  the  map  to  the  aviator's  changing  requirements. 

3.  Powerful  computational  capability.  The  increased  computational  capability  of  a  digital  map  system 
provides  the  basis  for  several  improvements  that  can  increase  the  interpretability  of  the  map  features. 
Examples  of  these  are  a)  using  shaded  elevation  bands  to  indicate  areas  where  the  surrounding  terrain 
is  equal  to,  higher  or  lower  than  the  altitude  at  which  the  aircraft  is  currently  flying;  b)  presenting  a 
shaded  relief  map  enhanced  by  contour  lines;  c)  displaying  the  areas  masked  from  visual  or  radar  ob¬ 
servation  given  known  or  likely  enemy  positions;  and  d)  constructing  perspective  views  to  familiarize 
the  aviator  with  the  terrain  as  it  will  be  seen  during  flight. 

4.  Increased  degree  of  interactivity  An  aviator  can  enter  information  such  as  map  annotations,  coordi¬ 
nates  of  objectives,  planned  routes,  etc.,  which  can  be  selected  at  will.  The  “intelligent"  nature  of  the 
system  can  permit  the  interrogation  by  the  aviator  to  determine  certain  characteristics  of  the  portrayed 
features,  such  as  tree  height  and  crown  cover.  Thus,  the  interactive  nature  of  the  system  can  remove 
some  of  the  natural  limits  to  the  aviator’s  decision-making  capabilities  and  permit  him  to  rapidly 
solve  complex  problems. 

As  a  result  of  his  research,  Rogers  (1985)  identified  several  desirable  functions  or  capabilities  for  a  computer¬ 
generated  digital  map.  These  include  the  ability  to: 

•  Present  different  map  scales  (e.g.,  1:50,000,  1:250,000); 

•  Show  different  map  areas  (e  g.,  near,  remote); 

•  Present  different  types  of  terrain  information  (e.g.,  contour  lines,  slope  shading); 

•  Present  different  map  orientations  (e.g.,  north-up,  track-up); 

•  Show  areas  of  masking  and  intervisibility  (e.g.,  clear  line-of-sight); 

•  Select  and  depict  a  wide  variety  of  features  (e.g.,  topographic,  tactical);  and 

•  Depict  elements  of  a  flight  plan  with  annotations  (e.g.,  flight  path,  waypoints). 

U.S.  Air  Force  Survey  of  Desirable  Digital  Map  Characteristics 

Rogers  and  Spiker  (1988)  conducted  a  survey  of  U.  S.  Air  Force  pilot  preference  for  digital  map  system  ca¬ 
pabilities  to  support  the  integration  of  a  digital  moving  map  system  into  advanced  high-performance  aircraft.  The 
project  approach  consisted  of  a  series  of  information-gathering  and  analytic  activities  that  included  a  literature  review, 
interviews  with  expert  aviators,  observations  of  aviator  task  performances,  surveys,  human  factors  engineering 
analyses,  and  perceptual  studies.  The  participants  were  experienced  Air  Force  active  duty  and  reserve  aviators. 

The  researchers’  findings  were  organized  under  a  systematic  outline  of  mission  phases  and  functions,  and 
identified  tasks  that  were  geographic  or  spatial  in  nature.  In  addition,  their  Findings  were  useful  for  defining  the 
specific  information  items  required  to  perform  the  tasks.  On  the  basis  of  their  findings,  candidate  digital  map 
applications  and  formats  were  developed  to  potentially  overcome  existing  interface  deficiencies. 

An  additional  group  of  pilots  evaluated  these  formats  using  a  questionnaire  survey.  The  survey  findings 
suggested  that  two  characteristics,  display  of  flight  leg  data,  and  perspective  view  of  terrain,  had  high  potential  utility 
and  thus  could  be  considered  to  be  extremely  advantageous.  In  addition  the  utility  ratings  of  several  other  character¬ 
istics  suggested  that  they  had  high  potential  payoff  and  value  for  enhancing  SA.  These  characteristics  included 
position  updating,  integrated  threat  display,  threat  circles  with  terrain  masking,  air-to-air  display,  and  terrain  avoid- 
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ance  display.  The  findings  from  the  survey  were  used  to  help  direct  technology  development  resources  toward  the 
application  of  the  most  critically  needed  capabilities. 

Human  Factors  Analysts  of  AH- 1 W  Moving  Map  Requirements 

Ruffner  and  Puccetti  (1996)  conducted  a  survey  of  previous  digital  moving  map  system  research  and 
of  existing  or  developmental  digital  map  systems  to  identify  desirable  capabilities  for  the  US  Marine  Corps 
AH-1 W  attack  hek.opter.  The  work  of  Rogers  and  his  colleagues,  described  previously,  was  summarized  in 
their  report.  One  of  the  programs  reviewed  by  Ruffner  and  Puccetti  was  the  P.AH-66  Comanche  digital  map 
development  program.  This  effort  represents  a  model  developmental  effort  from  a  human  factors  perspective 
in  which  SA  was  a  critical  design  driver  (Hamilton,  1993,  Hamilton  and  Metzler,  1992). 

The  Comanche  digital  map  was  designed  using  a  pilot-centered  approach  This  approach  that  was 
characterized  by  a  design  philosophy  in  which  data  for  SA  and  decision  making  were  brought  to  a  centralized 
display  location  in  a  manner  that  is  quickly  interpreted  relative  to  the  mission,  phase,  or  task  being  performed. 
In  addition,  the  map  information  was  provided  in  a  format  compatible  with  the  information  demands  of  the 
crew  and  organized  for  the  pilot’s  most  direct  comprehension  and  application  The  Comanche  digital  map  was 
designed  to  serve  as  a  mission  information  database  and  crew-aircraft  interface  as  well  as  a  primary  navigation 
aid. 

Based  on  their  findings,  Ruffner  and  Puccetti  (1996)  recommended  several  capabilities  that  should  be 
implemented  in  the  AH-1  W  digital  map  that  had  the  potential  for  enhancing  SA.  These  included:  (1)  allowing 
the  pilot  to  select  the  contour  line  interval  appropriate  for  the  mission  phase;  (2)  show  ing  areas  of  masking 
and  intervisibility  to  depict  the  likelihood  of  being  observed  or  detected;  (3)  allowing  north-up,  track-up,  and 
heading-up  map  orientations,  (4)  allowing  centered  or  decentered  location  of  ownship;  and  (5)  allowing 
slewing  of  the  map  to  another  selected  area. 

U.S.  Navy  Human  Factors  Digital  Map  Requirements  Study 

The  Navy  Research  Laboratory  (NRL)  recently  conducted  a  study  of  pilot  preferences  for  map  features  and 
capabilities  (Lohrenz  et  al.,  1997a;  Lohrenz  et  al.,  1997b).  The  study  consisted  of  one-on-one  aircrew  evaluations  of 
digital  maps  and  display  parameters  for  military  cockpits  The  researchers  guided  experienced  aircrew  through  task- 
structured  scenarios,  presented  a  variety  of  tactical  and  topographic  features  for  evaluation,  and  surveyed  participants’ 
preferences  based  on  their  platform  applications.  Representative  scenarios  were  presented  illustrating  candidate  map 
capabilities  such  as  (1)  map  positioning  (e.g.,  north-up,  track-up,  centered-decentered),  (2)  zooming  (e.g.,  zoom 
in/out,  continuous  versus  discrete  zoom),  (3)  presentation  of  terrain  elevation  data  (e.g.,  contour  lines,  plan  versus 
perspective  views),  (4)  map  overlay  data  (e.g.,  threat  location  and  range),  and  (5)  vector  map  displays.  Based  on 
their  findings,  several  capabilities  (e.g.,  map  overlay  data,  zooming)  are  being  incorporated  as  baseline  requirements 
in  the  TAMMAC  DMS  while  others  (e.g.,  perspective  view,  vector  map  displays)  are  considered  growth  capabilities 
and  may  be  added  at  a  later  date.  A  detailed  description  of  the  study  approach  and  findings  can  be  found  in  Lohrenz  et 
al.,  ( 1 977a).  Selected  study  findings  relevant  to  SA  are  summarized  in  the  following  paragraphs. 

Map  Positioning.  Most  pilots  preferred  track-up  orientation  over  a  north-up  orientation  for  improved  SA 
and  preferred  a  centered  aircraft  display  while  the  map  was  in  north-up  orientation.  The  pilots  judged  that  the 
bottom  of  the  screen  aircraft  position  results  in  loss  of  SA  behind  the  aircraft.  In  addition  the  pilots  consid¬ 
ered  the  north-up  orientation  to  be  disorienting  in  flight  but  good  for  waypoint  insertion 

Zooming  The  pilots  preferred  zooming  up  to  scale  of  the  next  chart  series,  then  switching  series  to 
maintain  SA.  They  judged  continuous  zoom  as  desirable  to  maintain  SA  in  a  controlled,  predictable,  and  fast 
manner.  The  pilots  judged  that  one-step  zoom  made  it  hard  to  keep  track  of  SA  but  that  zoom  out  supported 
maintaining  big-picture  SA.  They  also  preferred  easily  divisible  display  ranges  for  maintaining  SA. 

Terrain  Elevation  Data  There  was  no  strong  preference  for  a  two-dimensional  (2-D)  versus  a  three- 
dimensional  (3-D)  view,  with  the  judged  effectiveness  varying  with  terrain  elevation  display  mode  (e.g.,  chart 
data,  imagery).  The  pilots  considered  sun  angle  shading  a  good  SA  builder  for  flying  in  terrain  and  judged  that 
sun  angle  shading  would  increase  SA  and  enhance  their  ability  to  develop  an  evasive  plan  In  addition,  the 
pilots  judged  a  fixed  sun  angle  more  appropriate  than  a  dynamic  sun  angle  for  maintaining  SA.  Furthermore, 
contour  lines  were  preferred  more  by  helicopter  pilots  than  by  fixed-wing  tactical  pilots  for  maintaining  SA. 

Overlay  Data  The  pilots  judged  height  above  terrain  (HAT)  extremely  valuable  for  terrain  avoidance  and 
recommended  HAT  as  a  user-selectable  feature.  The  pilots  responded  favorably  to  the  clear  line-of-sight 
(CLOS)  capability.  The  pilots  preferred  threat  rings  for  displaying  intervisibility,  more  so  when  the  threat 
rings  were  overlaid  over  imagery  than  over  chart  data.  Furthermore,  the  pilots  preferred  translucent  overlays 
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instead  of  spokes  for  displaying  threat  information.  The  translucent  overlays  made  it  easier  to  see  the  under¬ 
lying  map  information. 

Vector  Maps.  Vector  are  rendered  from  individually  stoicJ  objects  such  as  points,  lines,  and  areas. 
The  pilots  favored  the  capability  of  vector  maps  for  keeping  text  upright  in  track-up  orientation  and  selectively 
decluttering  the  display.  They  judged  the  vector  map  capability  to  provide  enhanced  flexibility  and  display 
optimization  and  considered  it  good  for  building  and  sustaining  SA.  On  the  negative  side,  the  pilots  were 
concerned  that  vector  maps  might  add  complexity  and  increase  workload  There  were  also  concerns  that 
vector  maps  may  require  additional  pilot  training.  From  a  purely  technical  side,  vector  maps  are  likely  to 
require  additional  processing  and  capability  and  some  level  of  automated  cartography. 

Overall,  the  findings  from  the  NRL  study  suggested  that  pilots  favored  using  a  more  realistic  base-map  for 
SA,  but  overlaying  the  base-map  with  high  contrast,  mission-specific  features.  In  addition,  the  pilots  expressed  a 
strong  preference  for  keeping  the  map  as  simple  as  possible  for  more  rapid  assimilation  of  information  during  flight 
and  for  developing  SA,  Lastly,  the  pilots  recommended  putting  more  options  in  the  mission  planner  and  keeping  in¬ 
flight  options  to  a  minimum  for  the  greatest  SA  benefit  while  minimizing  in-flight  workload 

TAMMAC  Digital  Map  System  Features 

The  TAMMAC  DMS  will  provide  a  number  of  features  and  capabilities.  Some  baseline  features  will 
be  available  upon  Fielding  the  TAMMAC  system.  Other  growth  features  will  not  be  implemented  immediately. 
As  noted  previously,  many  of  these  features  are  described  in  Williams  (1998)  and,  therefore,  are  not  discussed 
in  detail  in  this  paper.  The  purpose  of  the  following  subsections  is  to  provide  the  reader  with  sufficient 
information  about  key  TAMMAC  DMS  baseline  and  growth  capabilities  to  serve  as  a  background  for  the 
ensuing  discussion  of  SA  guidelines  and  human  factors  research  and  design  issues. 

Baseline  Features 

TAMMAC  DMS  baseline  features  and  capabilities  include  the  following: 

•  Multiple  Display  Modes  (e.g.,  chart,  terrain  elevation,  imagery) 

•  Multiple  Display  Scales  (e.g.,  1 :50,000,  1 :250,000) 

•  Selectable  Map  Orientatioa'Reference  (e.g.,  north-up,  track-up) 

•  Overlay  Symbology  (e.g.,  ownship,  waypoints) 

•  Dual  Independent  Outputs  (e.g.,  pilot  and  copilot  crew  stations) 

•  Dynamic  Display  Overlays  (e.g.,  preplanned/pop-up  threats,  elevation  banding) 

•  Zooming  Capability  (e.g.,  zoom  in,  zoom  out) 

•  Selectable  Contour  Lines  Intervals  (e.g.,  50  feet,  100  feet) 

•  Trend  Dots  (indicating  aircraft  position  in  10,  20,  and  30  seconds) 

For  example,  Figure  1  illustrates  how  the  Dynamic  Display  Overlay  capability  might  be  implemented  using  elevation 
color  banding  and  threat  rings. 


Figure  1  Example  of  a  TAMMAC  DMS  baseline  feature:  Dynamic  Display  Overlay  show  ing  elevation  color 

banding  and  threat  rings. 
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Growth  Features 


Throughout  the  development  of  the  TAMMAC  system  several  advanced  display  features  were  considered 
that  offer  unique  capabilities  for  enhancing  and  maintaining  SA  but  will  not  be  implemented  immediately. 
TAMMAC  growth  features  include: 

•  3-D  Perspective  View 

•  Dynamic  Threat  Kings 

•  Declutterable  Vector  Map 

•  Picture-in  Picture  Inset  Window 

•  ln-flight  Mission  Re-planning 

•  Display  of  Map  Feature  Data 

•  Real-time  Imagery  in  the  Cockpit 

•  Three  Independent  Channels 

•  Predictive  Ground  Proximity  Warning  System  (GPWS) 

In  addition,  the  TAMMAC  system  will  be  able  to  incorporate  emerging  databases  from  the  National  Imagery 
and  Mapping  Agency,  (N1MA)  such  as  the  Vector  Vertical  Obstruction  Database  (VVOD).  Figure  2  illustrates 
an  example  of  the  picture-in-picture  growth  feature.  This  feature  would  allow  the  pilot  to  have  a  smaller 
picture  within  the  current  map  presentation  that  could  be  used  to  show  an  aerial  photograph,  a  data  frame  (e.g. 
a  list  of  threat  characteristics),  or  reconnaissance  imagery. 


Figure  2.  Example  of  a  TAMMAC  DMS  growth  feature:  Picture-in-picture  window 
showing  image  of  the  area  of  interest  in  the  inset  window. 

Situational-Awareness  Design  Guidelines 

Endsley  (1997b)  suggested  several  general  design  guidelines  for  creating  display  interfaces  that  en¬ 
hance  SA.  For  example,  cockpit  displays  should: 

•  Provide  information  that  is  processed  and  integrated  in  terms  of  SA  Level  2  and  3  needs, 

•  Provide  global  SA  along  with  goal-relevant  detailed  information, 

•  Present  information  in  terms  of  the  operator’s  major  goals. 


118 


•  Make  critical  cues  used  for  activating  mental  models  and  schemata  salient, 

•  Filter  extraneous  information  and  reduce  data  by  processing  and  integrating  low  level  data,  and 

•  Provide  system-generated  support  for  projecting  future  events  and  status  of  the  system. 

In  their  present  form,  the  guidelines  provide  useful  information  for  enhancing  SA  for  cockpit  display 
systems  in  general.  However,  they  need  to  be  tailored  and  selectively  applied  to  the  design  and  selection  of 
digital  moving  map  systems  features  and  capabilities.  For  example,  the  guidelines  should  provide  recommen¬ 
dations  for  color  and  shape  symbology  coding  for  the  depiction  of  threat  information  in  a  dynamic  overlay 
display  to  enhance  the  comprehension  of  threat  status  (SA  Level  2)  and  projected  ownship  vulnerability  (SA 
Level  3).  As  another  example,  the  guidelines  should  recommend  how  a  picture-in-picture  capability  could  be 
utilized  to  provide  global  SA  about  the  enroute  situation  while  providing  goal-relevant  detailed  information  to 
assist  the  aircrew  in  planning  their  tactics  for  the  terminal  objective  area, 

Human  Factors  Research  and  Design  Issues 

As  a  result  of  our  review  of  relevant  research  on  digital  moving  map  system  design  and  utilization,  vve 
identified  several  human  factors  research  and  design  issues  that  need  to  be  addressed  in  the  future  to  improve 
the  contribution  of  the  TAMMAC  DMS  to  enhancing  SA.  Specifically: 

•  What  are  the  individual  navigational,  and  tactical  tasks  that  require  the  development  and  maintenance  of  a 
high  level  of  SA?  What  are  the  global  and  local  SA  information  needs  of  these  tasks?  How  can  the  capa¬ 
bilities  in  the  TAMMAC  DMS  be  best  used  to  support  the  pilot’s  global  and  local  SA  information  needs? 

•  What  is  the  contribution  of  the  baseline  and  growth  TAMMAC  DMS  capabilities  to  promoting  and  main¬ 
taining  Level  1  (Perception),  Level  2  (Comprehension),  and  3  (Projection)  SA?  Which  growth  capabilities 
offer  the  most  potential  for  enhancing  SA  and  warrant  adaptation  in  the  future? 

•  What  is  the  best  way  to  apply  general  SA  guidelines  to  the  design  and  selection  of  TAMMAC  DMS  capa¬ 
bilities?  How  should  the  guidelines  be  modified  to  better  suit  the  requirements  specific  to  the  TAMMAC 
DMS? 

•  What  is  the  most  appropriate  way  to  measure  SA  for  the  TAMMAC  DMS  so  that  the  aircrew’s  global  and 
local  SA  information  needs  are  adequately  reflected? 

•  What  classes  (e.g.,  geographical,  tactical)  of  SA  elements  are  most  important  for  enhancing  SA  with  the 
TAMMAC  DMS?  What  is  the  best  way  to  use  or  enhance  DMS  capabilities  to  meet  the  information  needs 
of  these  SA  classes? 

Conclusions  and  Recommendations 

The  TAMMAC  digital  map  system  will  have  a  variety  of  features  and  capabilities  that  have  the  potential  for 
achieving  the  goal  of  enhancing  SA  and  aircrew  mission  effectiveness  without  further  burdening  pilot  workload.  The 
extent  to  which  this  potential  can  be  realized  will  depend  largely  on  the  successful  application  of  previous  digital 
moving  map  system  research  findings  and  the  thoughtful  tailoring  of  general  SA-oriented  design  guidelines  to  the 
design  and  selection  of  DMS  capabilities.  The  TAMMAC  system  provides  a  great  deal  of  flexibility  to  the  aircrew 
for  selecting  features  and  capabilities  to  support  their  specific  aircraft  mission.  Care  must  be  exercised  that  this 
flexibility  does  not  become  be  a  confributor  to  overall  aircrew  workload.  Accordingly,  more  specific  guidelines  need 
to  be  developed  and  validated  for  designing  and  selecting  DMS  features  to  promote  SA  for  different  aircraft  plat¬ 
forms. 

The  existing  TAMMAC  baseline  and  growth  requirements  and  capabilities  were  validated  based  on  user 
preferences  from  demonstrations  of  candidate  moving  map  capabilities  (e.g.,  Lohrenz  et  a!.,  1997a;  Lohrenz  et  al., 
1997b).  However,  there  is  evidence  that  preference  and  performance  are  not  always  consistent  (Bailey,  1993; 
Nielsen  and  Levy,  1994;  Wickens  and  Andre,  1994).  While  an  important  source  of  information  to  guide  the  design 
and  selection  of  digital  moving  map  system  features  and  capabilities,  preference  data  should  be  validated  using  part- 
task  or  full-task  simulation  scenarios  with  realistic  task  loadings  and  appropriate  performance  measures  that  are 
sensitive  to  critical  digital  moving  map  system  parameters.  Specifically,  advanced  capabilities  planned  for  the 
TAMMAC  DMS,  such  as  in-flight  mission  re-planning,  a  declutterable  vector  map,  and  picture-in-picture  should  be 
evaluated  in  user- performance  simulations.  The  results  of  these  simulations  could  be  used  to  optimize  the  map 
functions  and  capabilities  to  enhance  SA  while  minimize  the  impact  on  pilot  workload. 
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“Inside-Outside”  Spatial  Strategies  Employed  by  Pilots 
During  Shipboard  Operations 

LCDR  Frederick  R.  Patterson,  MSC,  USN 
Naval  Aerospace  Medical  Research  Laboratory 
Pensacola,  FL 

Introduction 

Aviation  training  specialists  have  established  that,  “the  pilot's  perception  of  orientation  serves  as  a 
backdrop  for  everything  he  dcx?s  with  the  aircraft  ”(1)  Based  on  this  philosophy,  efTorts  to  improve  flight  safety 
have  invariably  emphasized  enhancement  of  spatial  awareness  through  improved  training  and  display  technol¬ 
ogy.  Although  much  has  been  accomplished  toward  understanding  human  factors  related  to  these  issues,  loss  of 
spatial  and  situational  awareness  continues  to  plague  the  aviation  community.  Mishap  summaries  indicate 
sensory  or  cognitive  misinterpretations  are  responsible  for  as  much  as  76%  of  all  U.S.  military  aircraft  losses 
(2).  Each  year,  these  hazards  contribute  to  the  destruction  of  over  77  U.S  military  aircraft  and  result  in  the 
deaths  of  at  least  68  crew  members;  in  monetary  terms,  pilot  error  (caused  by  sensory  or  cognitive  mispercep¬ 
tions)  annually  cost  taxpayers  over  $360  million  in  lost  aviation  assets.  Surprisingly,  accident  investigations 
also  reveal  that  pilots  falling  prey  to  cognitive  hazards  are  not  typically  novice  aviators;  instead  they 
are  often  identified  as  highly  proficient  senior  pilots  with  ten  years  of  aviation  experience  and  over 
1500  hours  in  the  cockpit  (3). 

Aviation  accident  summaries  indicate  Navy  and  Marine  Corps  mishaps  (class  “A”)  have  a 
greater  than  50%  probability  of  occurring  at  sea  (4).  These  same  reports  further  suggest  many  over¬ 
water  losses  involve,  to  some  degree,  interaction  between  shipboard  and  aircraft  operations  (usually 
take-off  or  landing).  Since  loss  of  spatial  awareness  is  often  noted  as  a  reoccurring  cause  factor 
among  these  accidents,  it  seems  prudent  to  reevaluate  human  factor  design  issues  related  to  “at  sea” 
aviation  operations. 

Close-in  shipboard  flight  operations  require  that  pilots  maintain  spatial  awareness  by 
integrating  multiple  sensory-spatial  cues.  Following  shipboard  launch  and  departure,  pilots  usually 
orient  themselves  by  relying  on  traditional  aircraft  instruments  or  audio  cues  from  air  traffic  control¬ 
lers.  In  addition  to  these  references,  aviators  also  frequently  use  “outside”  visual  cues  for  direct  and 
unambiguous  spatial  information  relating  aircraft  attitude,  altitude,  and  airspeed.  During  approach 
and  landing,  two  additional  sources  of  spatial  information  play  a  critical  role  in  making  safe  shipboard 
recoveries:  these  two  factors  include  a  specially  trained  landing  signal  officer  (who  directly  observes 
the  aircraft  on  final  approach)  and  specially  designed  deck  mounted  displays  (ball)  which  provide 
direct  visual  information  relating  aircraft  position  relative  to  moving  ship 

With  nighttime  operations,  rotary-wing  pilots  may  also  utilize  night  vision  goggles  (NVG)  to 
enhance  their  “outside”  visual  spatial  cues.  In  addition  to  making  “outside”  cues  more  visible, 
updated  NVGs  systems  attempt  to  augment  spatial  awareness  by  overlaying  heads-up-display  (HUD) 
symbology  over  the  enhanced  NVG  imagery.  Although  both  old  and  new'  NVG  systems  were 
introduced  as  a  means  to  improve  spatial  awareness,  several  published  mishap  reports  suggest, 
“.... use  of  night  vision  devices  is  associated  with  increased  risk  of  spatial  disorientation"  (5).  In 
collaboration  with  this  assertion,  USMC  reports  indicate  over  a  recent  five  year  period  (1991-95) 
spatial  disorientation  (SDO)  was  the  primary  cause  factor  behind  15  of  their  33  helicopter  mishaps; 
many  of  which  involved  NVG  flight  (6).  During  a  comparable  five  year  span  (1987-92),  the  U.S. 
Army  reports  a  similarly  hazardous  situation  in  which  187  helicopter  losses  were  attributed  to  pilot 
disorientation  compounded  by  loss  of  situational  awareness  (5).  Of  additional  interest  are  documents 
indicating  most  disorientation  mishaps  do  not  typically  involve  “text  book”  causes  of  SDO  (which  are 
vestibular  or  visual  illusions);  instead,  U.S.  Army  investigators  have  found  cognitive  hazards  such  as 
distractions  or  failure  to  successfully  transition  to  instruments,  appear  as  primary  factors  behind  most 
NVG  and  non-NVG  “pilot  error”  incidents.  In  support  of  this  conclusion,  several  published  mishap 
summaries  indicate  visual  transition  from  “outside”  to  “inside”  is  a  factor  commonly  associated  with  aviation 
spatial  disorientation  (4,  7-9).  In  view  of  these  circumstances,  it  seems  prudent  to  further  examine  why 
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switching  from  “outside"  to  “inside"  the  aircraft  increases  risk  of  disorientation  and  loss  of  situational  aware¬ 
ness. 

Current  Design  Issues 

Since  an  aviator's  spatial  or  situational  awareness  is  largely  dependent  upon  how  well  he  interfaces 
with  cockpit  displays,  it  is  imperative  that  cockpit  designers  have  accurate  information  relating  which  display 
designs  are  most  sensory  compatible  and  therefore  most  resistant  to  cognitive  hazards.  Conventional  wisdom 
describing  aviation  spatial  awareness  assumes  pilots  view  a  moving  horizon  through  the  windscreen  {Figure 
1 }( 10- 1 2).  This  assumption  presupposes  head  alignment  with  the  vertical  cockpit  axis  during  both  visual 
(VMC)  and  inst^’ment  (IMC)  maneuvers.  Even  though  this  visual-spatial  paradigm  has  been  irxorporaled  with 
many  aircraft  designs,  its  accuracy  has  never  been  venfied. 


Figure  1 


Conventional  assumptions  related  to  pilot  spatial  awareness. 

Pilots  maintain  head  alignment  with  the  aircraft  during  both  IMC  and  VMC 
maneuvers. 

*5*  The  view  through  the  windscreen  is  a  moving  horizon  against  a  Fixed 
cockpit  perspective. 


In  contrast  to  the  conventional  design  theories  portrayed  in  Figure  I,  recent  human  factors  research  has 
identiFied  a  sensory-spatial  reflex  (opto-kinetic  cervical  reflex  or  “OKCR”)  that  causes  pilot  head  tilt  toward  the 
horizon  when  looking  “outside”  during  VMC  roll  or  pitch  maneuvers  {Figure  2}  (13-16)  Presumably,  this 
intuitive  head  tilt  response  improves  spatial  awareness  by  establishing  the  horizon  retinal  image  as  a  stabilized 
primary  visual-spatial  cue. 


Figure  2:  Blue  Angle  pilot  during  73  °  bank  in  F/A-18  aircraft. 
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OKCR  head  tilt  toward  the  horizon,  also  results  in  head  movement  away  from  the  cockpit  vertical 
axis.  An  important  outcome  of  this  movement  is  peripherally  viewed  cockpit  images  will  appear  to  move  with 
respect  to  the  retinaly  stabilized  horizon  view.  Presumably,  this  visual  perspective  aids  pilot  spatial  awareness 
by  establishing  peripheral  c^kpit  images  as  secondary  visual-spatial  cues  (visual-control  feedback)  that  move 
(on  the  retina)  in  the  same  direction  as  stick  control  inputs  (i.e.,  push  stick  left  -  peripheral  cockpit  images  move 
left)  (Figure  3).  By  adapting  this  visual-spatial  strategy,  pilots  are  able  to  achieve  motion  compatibility  between 
control  movement  and  visual  feedback;  this  beneficial  spatial  relationship  (“ principle  of  compatible  motion")  is 
known  to  reduce  both  mental  workload  and  episodes  of  disorientation  (11,17). 


Pilot  Spatial  Awareness  Models 


Revised  Paradigm 


Pnmaiy  Visual 
Spatial  Cue 
(stable  horizon) 


Secondan  Visual 
Spatial  Cue 
(moving  cockpil) 


Figure  3:  Pilot  spatial  awareness  theories  depicting  the  older  assumed  (conven¬ 
tional)  paradigm  and  the  recently  documented  (revised)  paradigm. 

Research  efforts  first  linking  OKCR  head  tilt  with  spatial  awareness  also  confirmed  that  sensory- 
spatial  reflexes  change  when  switching  from  VMC  to  IMC.  During  visual  transitions  from  “out-to-in,”  pilot’s 
consistently  realigned  their  heads  with  the  aircraft  vertical  axis  once  the  OKCR  stimulus  (outside  horizon)  was 
removed.  The  implications  of  this  sensory  transformation  are: 

*  An  IMC  pilot’s  view  of  the  cockpit  suddenly  becomes  stationary  as  his  view  of  the  (artifi¬ 
cial)  horizon  begins  moving. 

*  When  transitioning  from  VMC  to  IMC,  pilots  must  instantly  reverse  their  orientation  strat¬ 
egy  from  a  "fixed  horizon-moving  cockpit ”  to  a  “ moving  horton-fixed  cockpit '  perspec¬ 
tive 

*  In  conjunction  w  ith  reversal  of  spatial  cues  (out-to-in),  VMC-to-lMC  pilots  also  lose  sen¬ 
sory-spatial  compatibility  between  stick  control  motion  and  visual  feed  back  (i.e.,  push 
left-  visual  feedback  (display  symbol)  moves  right). 

Once  pilots  completely  transition  from  one  spatial  perspective  to  another,  they  are  usually  able 
to  control  the  aircraft  without  difficulty.  It  is  the  point  of  transition,  when  movement  of  spatial  cues 
(visual  feedback)  is  suddenly  reversed  and  control  compatibility  is  lost,  that  spatial  disorientation  and 
loss  of  situational  awareness  is  most  likely  to  occur  (13,  16,  18). 

The  most  common  spatial  disorientation  error  associated  with  instrument  transition  is  mistaking 
horizon  symbology  motion  (located  on  the  attitude  indicator)  as  relative  movement  of  the  aircraft  w  ings  This 
misinterpretation,  referred  to  as  control  reversal  error ,  becomes  apparent  when  a  pilot  inadvertently  executes  a 
control  input  resulting  in  aircraft  movement  opposite  what  was  intended.  Early  research  evaluating  frequency 
of  control  reversal  error  suggested  pilots  commit  reversal  errors  approximately  7%  of  the  time,  during  relatively 
eas\  transitions  (18).  More  current  evaluations  aimed  at  evaluating  control  problems  with  difficult  VMC-1MC 
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transitions  (such  as  formation  flying  or  unusual  attitude  recovery)  report  much  higher  reversal  error  rates 
ranging  between  25  and  65  %  (1 3,  16). 

In  respect  to  head-mounted-displays  (HMD),  which  include  NVGs,  recently  completed  U.S.  Army  re¬ 
search  found  pilots  using  NVGs  experienced  the  same  reflexive  (OKCR)  head  tilt  as  reported  with  daytime 
VMC  (16).  The  inference  of  this  research  is  pilots  flying  at  night  (with  NVGs)  apply  the  same  stabilized 
horizon-moving  cockpit  strategy  as  normally  used  during  day  VMC.  However,  of  concern  with  NVG  flight  is 
the  fact  that  peripherally  viewed  cockpit  images,  which  provide  critical  control-visual  feedback,  fall  beyond  the 
c  lhanced  NVG  (40°)  fi  d  of  view;  subsequently,  these  secondary  spatial  cues  become  significantly  degraded 
by  the  low  peripheral  light  levels.  In  view  of  these  circumstances,  the  following  sensory-spatial  factors  are 
germane  to  NVG  safety  and  flight  performance: 

*  During  certain  phases  of  flight,  particularly  during  execution  of  high  angle  of  bank  maneu¬ 
vers,  secondary  spatial  cues  become  seriously  degraded  or  made  altogether  invisible  by  the  subsequent 
loss  of  peripheral  visual  acuity. 

*  Loss  or  degradation  of  these  secondary  references  can  be  expected  to  significantly  reduce  pilot 
spatial  awareness  while  at  the  same  time  causing  an  increase  in  mental  workload. 

In  addition  to  dealing  with  reduced  primary'  and  secondary  visual-spatial  cues,  pilots  flying  with 
NVGs  may  also  experience  the  need  to  suddenly  transition  to  instruments  during  unexpected  1MC  or 
brownout/whiteout  conditions.  Depending  on  the  aircraft  type  and  NVG  series,  pilots  who  lose  outside 
cues  may  transition  to  one  of  two  available  instrument  attitude  references.  Currently,  the  most  common 
method  of  NVG-to-instrument  transition  is  to  look  under  the  goggles  at  the  head-down  displays  located 
on  the  forward  instrument  panel.  From  a  cognitive  standpoint  this  method  is  comparable  to  going  1MC 
during  daylight  conditions  in  that  it  also  causes  loss  of  secondary  spatial  cues  and  necessitates  a  reversal 
of  spatial  strategies. 

A  second,  less  widely  available  method,  is  to  focus  attention  on  HUD  attitude  symbology  superim¬ 
posed  over  the  NVG  field-of-view  (Figure  3).  This  newly  introduced  technology,  brought  on  line  by  rapid 
prototyping  methods  (19),  consists  of  traditional  HUD  symbology  overlaid  onto  standard  AN  VIS  NVG  images. 
Although  this  form  of  sensory  fusion  first  appeared  as  a  means  of  improving  spatial  awareness,  there  exists 
some  concern  regarding  the  sensory-spatial  compatibility  of  this  system  These  concerns  center  around  the  fact 
that  HUD  symbology,  which  was  originally  designed  as  a  fixed  forward  looking  reference,  may  not  be  well 
suited  as  an  off-axis  (head  turned)  visual  attitude  reference. 


Figure  3:  HUD  symbology  superimposed  with  NVG  images  (AN/AVS-7) 

Human  factor  engineers  have  determined  current  “fixed  head”  NVG-HUDs  have  an  orientational  per¬ 
spective  that  is  difficult  to  interpret  and  prone  toward  inducing  spatial  disorientation  (20-23).  The  primary 
problem  associated  with  this  fixed  head  perspective  is  NVG-HUD  symbology  moves  with  the  head  during 
voluntary  or  reflexive  (OKCR)  head  movements.  Under  these  conditions,  orientation  of  HUD  symbology 
becomes  dependent  on  changing  head  positions;  as  a  result,  the  horizon  line  symbol  becomes  out  of  alignment 
with  the  real  world  and  the  aircraft  waterline  symbol  is  not  aligned  relative  to  the  airframe,  pilot,  or  the  actual 
horizon  (Figure  4).  In  essence,  during  NVG-HUD  flight  the  pilot’s  attitude  display  symbology  will,  most  of  the 
time,  remain  significantly  out  of  alignment  with  the  real  world  horizon  (primary  spatial  cue)  and  aircraft  axis 
(secondary  spatial  cue).  As  suggested  by  the  cited  references,  this  mismatch  between  attitude  display  sym¬ 
bology  and  the  real  world  environment  leads  to  a  significant  increase  in  mental  workload  (with  potential  for  task 
saturation)  and  an  increase  likelihood  of  experiencing  spatial  disorientation  or  loss  of  situational  awareness. 
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Figure  4:  NVG-HUD  images  indicating  loss  of  alignment  that  occurs  between  horizon 
symbology  and  real  world  horizon  visual  reference. 


Conclusions 

Current  design  standards  for  NVG,  HUD,  HMD  and  head-down  (HDD)  symbology  require  pilots  to 
make  use  of  an  instrument  spatial  strategy  that  is  quite  different  (opposite)  from  the  one  employed  when  looking 
outside  the  cockpit.  As  a  result,  pilots  transitioning  between  “inside”  and  “outside”  during  high  workload 
conditions  (such  as  shipboard  approaches)  are  likely  predisposed  toward  increased  mental  workloads  or 
cognitive  hazards,  which  may  contribute  toward  spatial  disorientation  and  loss  of  situational  awareness. 

Recommendations 

In  order  to  make  the  cockpit  a  more  human-sensory  compatible  environment,  and  thereby  improve 
spatial  or  situational  awareness,  the  following  areas  of  research  and  training  are  proposed: 

1 .  Investigate  the  cause  and  prevention  of  disorientation  and  loss  of  situational  awareness  related  to 

sensory-display  incompatibility. 

2 .  Develop  sensory  compatible  display  syvibology  for  HMDs  (including  NVG)  systems 

3.  Develop  virtual  reality  (HMD-VR)  or  monitor  training  software  that  emphasizes  cognitive  haz¬ 
ards,  such  as  semory'-spatial  incompatibility,  reversal  error,  task  saturation,  and  visual  problems 

Disclaimer:  The  views  expressed  in  this  article  are  those  of  the  authors,  and  do  not  necessarily  reflect  the 
official  policy  or  position  of  the  Department  of  the  Navy,  Department  of  Defense,  nor  the  U.S.  Government 
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Supporting  Situation  Awareness  in  Tactical  Air  Environments 
Using  3-D  Auditory  Displays 


\V.  Todd  Nelson  and  Richard  L.  McKinley 
Air  Force  Research  Laboratory 
Human  Effectiveness  Directorate 
Wright-Patterson  Air  Force  Base,  OH 

The  purpose  of  this  paper  is  to  consider  the  potential  utility  of  3-D  auditory  interface  technology  to  support 
situation  awareness  (SA)  in  tactical  aviation  environments.  Indeed,  advances  in  computer  and  virtual  environ¬ 
ment  technology  over  the  past  decade  have  permitted  the  development  of  3-D  auditory  systems  that  are  capable 
of  generating  effective  spatial  auditory  cues.  Moreover,  numerous  laboratory  investigations  have  demonstrated 
the  efficacy  of  this  technology  for  enhancing  performance  and  reducing  workload  on  tasks  that  are  relevant  to 
tactical  air  environments  (i.e.,  virtual  3-D  auditory  displays  have  been  shown  to  enhance  performance  on  visual 
target  detection  and  identification  tasks  using  projection  and  helmet-mounted  displays).  Yet,  empirical  evidence 
demonstrating  the  effect  of  v  irtual  3-D  audio  technology  on  improving  pilot's  SA  has  been  extremely  sparse 

In  our  opinion,  this  state  of  affairs  is  primarily  due  to  the  elusive  nature  of  the  SA  construct,  and  not  to 
limitations  in  the  functional  utility  of  virtual  auditory  interfaces.  To  be  sure,  some  researchers  have  recently 
questioned,  or  in  some  cases  denounced,  the  concept  of  SA  as  autological  and  purely  descriptive.  We  believe, 
however,  that  the  concept  of  SA  is  useful,  especially  for  human  factors  researchers  who  are  designing  interfaces 
for  tactical  aircraft 

In  formulating  our  position,  we  will  adopt  a  view  of  SA  articulated  by  Smith  and  Hancock  (1995);  that  is, 
we  maintain  that  tactical  SA  exists  in  the  invariant  interaction  between  crew  members  and  the  tactical  air 
environment.  Within  this  framework,  tactical  SA  specifies  what  crewmembers  must  know'  to  successfully 
resolve  the  challenges  of  tactical  air  environments.  Accordingly,  it  will  be  important  to  consider  (I)  whether 
current  virtual  3-D  auditory  technology  has  matured  sufficiently  to  specify  the  invariant  information  conveyed 
by  tactical  situation  awareness;  and  (2)  how  best  to  design  virtual  aural  interfaces  so  that  they  transfer  the  cntical 
aspects  of  the  tactical  air  environment. 

Accordingly,  we  will  argue  that  to  the  extent  which  virtual  3-D  auditory  displays  can  be  designed  to  convey 
meaningful  spatial  information,  they  will  support  tactical  SA.  Viewing  SA  in  this  way,  we  will  propose  that 
spatial  auditory  displays  may  be  particularly  useful  for  augmenting  interfaces  used  for  specific  navigation, 
spatial  orientation,  and  Instrument  Flight  Rules  (1FR)  and  Instrument  Meteorological  Conditions  (IMC) 
applications. 


(Reprint  of  executive  summary;  formal  paper  not  available.) 
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Spatial  Disorientation  in  U.S.  Army  Rotary  Wing  Aircraft 


COL  Edwin  A.  Murdock,  USA 

U.S.  Army  Safety  Center 
Ft.  Rucker,  AL 

Spatial  Disorientation  (SD)  is  currently  a  major  hazard  in  U.S.  Army  Rotary  Wing  (RW)  Operations. 
A  study  of  Class  A-C  Aviation  Accidents  by  the  U.S.  Army  Safety  Center  (USASC)  and  the  U.S.  Army 
Acromedical  Research  Laboratory  (USAARL)  noted  that  SD  played  a  major  role  in  32%  of  Class  A-C 
accidents  and  was  responsible  for  60%  of  Army  RW  aviation  fatalities  during  the  1987  -  1992  timeframe  A 
follow-up  study  by  USAARL  adding  3  1/2  more  years  data  noted  that  SD  played  a  major  role  in  30%  of  Class 
A-C  RW  accidents  and  was  responsible  for  59%  of  aviation  fatalities  for  the  1987  -  1995  timeframe.  As  a  result 
of  these  studies,  recommendations  for  controls  to  address  this  problem  were  developed.  Efforts  in  education  of 
aircrew,  adoption  of  the  British  SD  sortie,  adaptation  of  simulators  to  include  SD  accident  scenarios,  and  efforts 
to  recommend  technology  controls  will  be  presented. 


(Reprint  of  executive  summary;  formal  paper  not  available.) 
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NEUROPSYCHOLOGICAL  GUIDELINES 
FOR  AIRCRAFT  CONTROL  STA  TIONS 


Fred  H.  Previc 

Air  Force  Research  Laboratory 
Flight  Motion  Effects  Branch 
Brooks  Air  Force  Base,  TX 


Introduction 

Ideally,  workstations  used  in  controlling  aircraft  should  allow  the  operator  to  process  information  according  to 
the  way  that  the  human  brain  naturally  does  so.  In  our  everyday  existence,  humans  ordinarily  process  and  act  upon  an 
enormous  amount  of  information,  much  of  it  preconscious.  For  instance,  we  continuously  monitor  and  update  our 
present  and  future  geographical  location,  local  meteorological  conditions,  and  the  significance  of  various  other  external 
objects  and  events,  along  with  our  posture  in  space,  nutritional  status,  and  other  information  emanating  from  our  bodily 
senses.  It  is  arguable  that,  in  our  normal  terrestrial  existence,  we  process  much  more  information  than  do  even  pilots  of 
high-performance  aircraft,  despite  the  fact  that  the  latter  are  task-saturated  much  of  their  time.  Moreover,  our  natural 
information  processing  occurs  at  high  speeds— e.g.,  the  optical  flow  speeds  associated  with  vehicular  locomotion  on  earth 
are  often  of  considerably  higher  velocity  than  that  encountered  by  pilots  in  aircraft.  The  fundamental  problem  facing 
pilots  and  other  aircraft  operators,  then,  is  not  the  amount  and  speed  of  their  required  information  processing  but  rather 
the  quality  and  intuitiveness  of  the  information  provided  to  them. 

Unfortunately,  it  is  impossible  to  duplicate  much  of  the  information  processing  that  we  naturally  engage  in  as 
we  move  about  on  the  face  of  the  earth.  For  one,  signals  concerning  the  status  of  our  motion  platform  (e.g.,  the  body) 
are  transmitted  by  means  of  interoceptive  sensory  systems  that  cannot  be  used  to  infer  the  status  of  the  aircraft  in  flight. 
Second,  nonvisual  sensory  systems  (e.g.,  the  vestibular  and  somatosensory  ones)  that  provide  rapid  and  accurate 
information  concerning  our  position  in  space  during  natural  movements  are  rendered  unreliable  in  flight  due  to  the 
existence  of  gravitoinertial  force  vectors  that  deviate  from  the  direction  of  true  gravity.  Third,  the  full  three-dimensional 
(3-D)  extent  of  the  natural  visual  world  cannot  be  duplicated  within  the  finite  confines  of  the  aircraft  or  any  other  portable 
platform  housing  an  aircraft  control  display  .  Finally,  the  motor  systems  used  in  locomoting  on  earth  (i.e.,  the  trunk  and 
lower  limbs)  cannot  be  as  easily  utilized  in  controlling  the  motion  of  the  aircraft,  thereby  forcing  operators  to  over-rely  on 
manual-control  systems  that  are  ordinarily  used  in  reaching,  object  manipulation,  and  other  sophisticated  visuomotor 
interactions  unrelated  to  locomotion.  Although  an  ideal  aircraft  control  station  should  attempt  to  duplicate  the  normal 
terrestrial  environment  whenever  possible,  it  should  to  an  even  greater  extent  be  commensurate  with  the  mental  nuxiels 
used  by  different  brain  systems  involved  in  carrying  our  everyday  perceptual -motor  interactions.  A  recent  neuropsy¬ 
chological  model  of  how  humans  interact  with  our  3-D  spatial  environment  (Previc,  1998)  may  prove  especially  fruitful 
in  this  regard.  This  model,  shown  in  Figure  I,  proposes  that  four  major  brain  systems  mediate  our  perceptual-motor 
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Figure  1.  The  neuropsychology  of  3-D  space.  The  four  behavioral  realms  are  shown 
at  left,  while  the  four  cortical  systems  corresponding  to  each  are  shown  at  right  From 


interactions  in  3-D  space.  A  dorsolateral  cortical  system  subserves  our  interactions  in  pcripersonal  space,  in  which 
reaching  and  object  manipulation  occur.  A  ventrolateral  cortical  system  is  used  in  focal  extrapcrsonal  space  to  search  for 
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and  recognize  objects  and  other  detailed  information  (e.g.,  alphanumeric  symbology).  A  ventromedial  cortical  system  is 
used  to  navigate  and  orient  in  action  extrapersonal  space,  which  is  defined  as  our  topographical  (geographical)  environ¬ 
ment.  Finally,  a  dorsomedial  system  is  used  to  orient  in  and  locomote  about  ambient  extrapersonal  space,  which  is 
defined  in  earth-fixed  or  gravitational  coordinates. 

As  further  delineated  in  Table  1,  each  of  these  four  brain  systems  interacts  with  its  own  region  of  the  3-D 
world,  has  a  unique  combination  of  predominant  sensory  and  motor  systems,  and  uses  a  unique  coordinate  system 

Tabic  1 
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for  carrying  out  perceptual-motor  interactions  within  that  space.  For  example,  our  peripersonal  system  is  I) 
biased  to  the  lower,  proximal  visual  field,  2)  relies  on  visual,  vestibular,  and  somatosensory  sensory  inputs,  3) 
uses  upper-limb  movements  as  its  major  motoric  instrument,  and  4)  has  a  spatial  coordinate  system  that  is  centered 
around  the  upper  limbs  and  torso.  Conversely,  the  action  extrapersonal  system  is  1)  biased  to  the  upper,  distant 
visual  field,  2)  relies  on  visual  and  auditory  sensory  inputs,  3)  uses  primarily  horizontal  head  movements  in 
orienting  and  navigating,  and  4)  has  its  coordinate  system  centered  around  the  position  of  gaze. 

Before  proceeding  to  discuss  how  aircraft  control  stations  can  be  designed  commensurate  with  the 
operations  performed  in  each  realm  of  3-D  space,  two  general  points  will  be  noted  about  the  ecology  of  our  3-D 
world.  First,  one  of  the  factors  that  helps  to  unify  the  four  3-D  realms  is  the  fact  that  all  rely  on  vision  as  their 
primary  sensory  system,  although  the  type  of  visual  processing  that  predominates  in  each  realm  is  not  always  the 
same  (e.g.,  “global”  motion  processing  is  used  extensively  in  peripersonal  space,  whereas  sustained  “local”  visual 
processing  is  relied  on  in  focal  extrapersonal  space).  Hence,  purely  visual  displays  should  be  able  to  provide  a 
good  awareness  of  the  position  of  the  aircraft  in  relationship  to  its  surrounding  space.  Second,  one  of  the  most 
fundamental  features  of  the  visual  environment  is  that  the  ground  plane  slopes  up  and  away  from  the  observer; 
thus,  the  more  proximal  and  distal  portions  of  the  visual  world  are  located  in  the  lower  and  upper  visual  fields, 
respectively.  Not  surprisingly,  the  preferred  viewing  angle  for  a  given  display  becomes  elevated  as  the  distance 
to  it  increases  (Hill  &  Kroemer,  1986),  just  as  optical  vergence  and  accommodation  becomes  more  distant  with 
increasing  elevation  of  our  eyes  (Heuer  &  Owens,  1989)  Any  truly  well-designed  workstation-whether  used  for 
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controlling  aircraft  or  not-should  embrace  this  cardinal  feature  of  our  3-D  environment 

The  next  section  of  this  paper  will  illustrate  how  a  neuropsychological  understanding  of  how  humans  interact 
with  our  3-D  spatial  environment  can  lead  to  specific  predictions  concerning  the  location  and  features  of  particular 
displays  and  controls. 

3-D  Spatial  Realms:  Functional  Characteristics  and  Relevance  for  Aircraft-Control  Stations 

Peripersonal.  The  primary  functions  carried  out  in  peripersonal  space  are  reaching  for  objects  and  other 
manipulations  that,  at  least  in  nonhuman  primates,  are  closely  tied  to  consummatory  behavior.  The  primary  sensory 
inputs  (visual,  somatosensory  /proprioceptive,  and  vestibular)  are  used  to  align  gaze  and  limb  position  with  the  object, 
while  the  primary  motor  outputs  are  designed  either  to  grasp  and  manipulate  the  object  using  the  arm  and  hand  or  to  track 
(using  head  and  smooth  ocular  movements)  the  arm  and  hand  as  they  reach  for  the  object  and  as  it  is  manipulated  (Previc, 
1998).  The  visual  specializations  of  our  peripersonal  system  lie  in  I)  “global”  motion  processing,  which  is  necessitated 
by  the  impossibility  of  processing  the  rapid  and  optically  degraded  motion  usually  found  in  this  realm  by  “local”  contour 
mechanisms,  and  2)  “coordinate"  processing,  which  is  required  of  the  precise  visuospatial  distance  judgments  needed  to 
reach  and  grasp  objects.  Because  of  the  close  affinity'  with  feeding,  peripersonal  behaviors  are  also  linked  to  homeostatic 
signals  concerning  the  state  of  the  body  itself. 

In  tapping  into  the  natural  proclivities  of  our  peripersonal  brain  system,  aircraft-control  stations  should  facilitate 
the  use  of  “analogue”  (trajectory-  or  movement-based)  visuomanual  mechanisms  in  directly  interacting  with  visual 
displays.  This  manual  behavior  should  occur  in  the  normal  location  of  peripersonal  activity-namely,  the  mid-to-lowcr 
portion  of  the  station  and  within  +/-30  deg  of  the  midline  of  the  control  station.  In  fact,  most  aircraft  control-stations 
grossly  violate  this  guideline  by  I )  requiring  that  the  manual  system  be  used  primarily  for  controlling  the  aircraft  in  space 
(normally  a  task  of  our  whole-body  locomotory  systems),  2)  forcing  manual  behavior  to  be  carried  out  well  outside  of  its 
normal  range  (e.g.,  control  panels  that  extend  90  degrees  or  more  to  the  side  of  the  cockpit  or  above  the  pilot),  and  3) 
using  only  “digital”  responses  (e.g.,  “button-pressing”)  to  interact  with  arbitrarily  located  switch  settings  whose  combina¬ 
tions  run  into  the  thousands.  The  first  violation  can  be  solved  by  allowing  aircraft  control  to  be  partly  assumed  by  the 
feet  (see  later  section),  which  would  thereby  free  up  at  least  one  hand  for  purely  visuomanual  activity  .  The  second  and 
third  violations  could  be  dealt  with  by  minimizing  the  overall  number  of  switches  and  locating  them  in  normal  reaching 
space.  One  way  to  do  this  would  be  to  use  touch-screen  or  cursor  (mouse)  technologies  that  facilitate  direct  and/or 
“analogue”  interactions  with  the  information  that  is  being  manipulated  at  a  particular  moment.  Alternatively,  one  could 
control  displays  by  means  of  voluntary'  saccadic  eye  movements,  voluntary  head  movements,  or  vocal  commands. 
However,  the  saccadic  and  vocal  systems  are  not  normally  used  in  peripersonal  space,  and  both  are  subject  to  signal-to- 
noise  problems  in  psychologically  stressful  (e  g.,  high-workload)  and  environmentally  stressful  (e  g.,  noisy)  aircraft- 
control  environments.  By  comparison,  smooth  head  movements  are  naturally  used  in  conjunction  with  smooth  pursuit 
and  vergence  movements  in  peripersonal  space  and  are  being  used  in  specific  applications  such  as  tracking  of  out-the- 
window  targets  using  helmet-mounted  displays.  However,  head  movements  are  more  likely  to  be  used  in  certain 
tracking  directions  than  others  (Andre-Deshays  et  al.,  1993),  are  supplemented  under  natural  circumstances  by  eye 
movements,  and  can  create  disorientation  at  high  G-levels  (Gilson  et  al.,  1973);  hence,  optimal  helmet-mounted  tracking 
systems  should  not  rely  on  head-tracking  alone. 

Because  global  motion  and  coordinate  processing  are  important  in  peripersonal  visuomanual  activity,  they  are 
also  tied  to  the  proximal  lower  visual  Field  (Previc,  1998).  Thus,  another  neuropsychological  guideline  concerning 
peripersonal  operations  would  be  to  place  many  moving  or  coord  in  ate- type  displays  (e.g.,  pictorial  or  nonverbal)  related 
to  weapons  and  aircraft  status  in  the  mid-to-lower  portion  of  the  control  station.  Integrated  graphical  displays  have  not 
only  proven  superior  in  many  psychological  studies  (Bennett  &  Flach,  1992)  but  have  met  with  good  acceptance  by  pilots 
(Way  et  al.,  1984)  and  are  widely  used  in  commercial  products.1  The  use  of  transient  warning  cues  should  also  be 
effective  in  this  region  of  the  control  station,  given  the  natural  role  of  transient  visual  cues  in  peripersonal  processing 
(Previc,  1998). 

Focal  Extrapersonal.  The  main  function  of  focal  extrapersonal  operations  is  to  search  for  objects  and  other 
forms  and  to  recognize  them.  The  focal  extrapersonal  system  is  the  primary  one  involved  in  processing  alphanumeric 


Although  many  piclorial  displays  involve,  lo  some  extern,  objccl-based  recognilion  processes  lhai  are  normally  applied  in  focal- 
cxirapersonal  space,  a  knowledge  of  aircraft  status  itself  is  (like  bodily  physiological  staius)  a  fundamcnially  peripersonal  activily  Anolher 
reason  for  placing  ihc  aircraft  sialus  displays  in  the  lower  portion  of  lhe  console  is  lhai  lhey  do  not  have  lo  be  as  continuously  monitored  as 
critical  alphanumeric  and  global  silualion  informaiion  lhai  is  much  more  advantageouslv  presented  in  ihc  upper  portion  of  lhe  console  (sec 
ensuing  discussion). 
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information  from  control  displays.  Generally,  its  “retinotopic”  coordinate  system  is  tied  to  the  position  of  the  eyes  in  the 
orbit  and  moves  with  the  eyes  in  depth.  The  most  important  sensory  input  by  far  is  vision,  while  the  main  motoric 
instrument  used  by  the  focal  extrapersonal  system  is  saccadic  eye  movements. 

The  most  critical  control-station  guidelines  based  on  an  understanding  of  focal-extrapersonal  operations 
relates  to  the  size  and  positioning  of  *arious  display  elements.  Focal  extrapersonal  operations  are  limited  to  the 
central  30  degrees  of  the  visual  Field,  wherein  our  visual  acuity  is  adequate  to  support  the  high-resolution  ‘local” 
contour  analysis  required  for  alphanumeric  processing  (Previc,  1998).  Even  in  an  uncluttered  visual  Field,  targets 
presented  more  than  15  degrees  away  from  the  Fixation  point  cannot  reliably  be  detected  without  moving  the  eyes. 
In  a  cluttered  visual  Field,  eye  movements  may  be  required  when  local  form  information  exceeds  more  than  a  few 
degrees  in  eccentricity,  and  head  movements  may  occur  when  the  target  exceeds  15  degrees  in  eccentricity  (Bahill 
et  al.,  1975).  Based  on  visual  search  studies  (Previc,  1996;  Previc  &  Blume,  1993),  the  range  of  focal  extraper¬ 
sonal  vision  in  most  individuals  is  extended  in  the  upper  visual  Field  (particularly  the  upper-right  quadrant)  and 
limited  in  the  lower  Field  (especially  the  lower-left  quadrant)  (  Figure  2).  The  upper-Field  bias  of  the  focal 
extrapersonal  system  is  attributable  to  1)  the 
fact  that  focal-mode  operations  are  usually 
carried  out  at  a  greater  distance  from  the 
individual  than  are  peripersonal  operations, 
and  2)  the  need  to  offset  the  latter’s  lower- 
field  attentional  and  oculomotor  biases. 

Thus,  a  given  display  should  not  exceed  30 
degrees  in  diameter  (to  avoid  head  move¬ 
ments)  and  individual  readouts  on  the  display 
panel  should  not  exceed  10  degrees  (to  allow 
the  information  to  be  processed  in  a  single 
Fixation).  The  most  critical  readouts  should 
be  located  above  the  center  of  the  display  and 
toward  the  upper  portion  of  the  control- 
station.  Whereas  transiently  presented 
information  may  better  simulate  the  natural 
visual  cues  found  in  peripersonal  space,  such 
a  mode  of  presentation  is  less  desirable  in 
focal  extrapersonal  operations  because  high- 
resolution  information  processing  is  degraded 
when  presented  transiently.2  On  the  other  hand,  color  is  a  salient  cue  for  the  focal-extrapersonal  brain  pathways 
and  should  be  used  in  depicting  solid  display  objects;  however,  color  is  less  desirable  for  line  and  character 
symbols  (Way  et  al.,  1984),  given  the  poorer  spatial  resolution  for  purely  chromatic  stimuli  as  opposed  to 
achromatic  stimuli. 

Action  Exlrapersonai  Operations  in  this  realm  are  associated  with  navigation  and  other  types  of  orientation  in 
topographically  defined  space.  Although  the  coordinate  system  used  in  topographical  space  is  ultimately  based  on  the 
position  of  gaze  (i.e.,  viewpoint),  action  extrapersonal  operations  are  most  closely  linked  to  movements  of  the  head, 
which  generally  precede  changes  in  heading  during  locomotion  (Grasso  et  al.,  1996)  and  predominate  in  orienting  to 
targets  beyond  30-40  degrees  in  eccentricity  (Barnes,  1979).  The  two  major  sensory  systems  used  in  action  extraper¬ 
sonal  space  in  humans— the  visual  and  auditory— require  the  integration  of  head  position  signals  to  deFine  target  orientation 
in  relationship  to  viewpoint  in  topographical  space.  The  predominant  role  of  head  movements  in  topographical  space  is 
consistent  with  the  increasing  use  of  helmet-mounted  systems  for  localizing  tactical  targets. 

Although  topographical  visual  space  can  extend  to  the  edges  of  the  visual  Field  and  topographical 
auditory  space  can  completely  encompass  the  individual,  our  mental  representation  of  such  space  is  actually  quite 
compressed,  especially  in  its  nonvisible  portions  (Beer,  1993).  Studies  of  scene  memory,  for  example,  have 
shown  that  immediate  memory  representations  are  severely  distorted  such  that  a  wide-angle,  gaze-centered 
prototypical  view  is  assumed  (Figure  3).  Thus,  the  global  situation  display  in  an  aircraft  that  is  designed  to 
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Figure  2.  Visual  search pcrjormance  as  a 
function  of  target  location  in  the  four  quadrants 
of  the  visual  field.  Data  are  expressed  as 
percentage  of  trials  in  which  an  eye  movement 
was  not  needed  to  locate  the  target  (above)  and 

2  One  example  of  target-stabilization  is  the  Forward  Looking  Infrared  Imaging  (FUR)  display,  which  helps  the  borcsight  lo  be  placed  on  the 
target  by  stabilizing  the  latter 


138 


present  geographical,  tactical,  and  other  topographical  information  should  probably  adopt  a  wide-angle  perspec¬ 
tive  based  on  how  the  pilot  would  normally  view  the  world-from  above  and  behind.  Because  our  action  extraper¬ 
sonal  system  apparently  moves  in  yaw  but  not  pitch  and  roll-as  evidenced  by  the  sensitivity  of  parahippocampal 
neurons  involved  in  topographical  memory  to  horizontal  head  movements  but  not  to  pitch  and  roll  head  tilts 
(Taube  et  al.,  1 996) — the  global  situation  display  should  ideally  adopt  a  track-up  (variable-heading)  depiction  that 
shows  a  stable  horizon  In  fact,  a  track-up  (variable-heading)  depiction  seems  to  be  most  consistent  with  our 
mental  models  of  heading  changes  during  forward  locomotion  (Shepard  &  Hurwitz,  1984)  and  is  generally 
preferred  by  pilots  and  other  operators  (Wickens  et  al.,  1996). 


Original 


Figure  3.  Wide-angle  topographical  memory  representation  in  humans.  Original  illustra¬ 
tion  is  shown  at  left ;  reproduced  drawing  from  memory  is  shown  at  right.  (From  Intrauh 


Like  focal  extrapersonal  space,  action  extrapersonal  space  is  biased  toward  the  upper,  distant  portion  of 
the  visual  world  One  way  to  simulate  both  the  wide-angle  and  prototypical  distance  of  the  mental  representa¬ 
tions  associated  with  this  realm  is  to  use  a  planoconvex  collimating  (e.g.,  Fresnel)  lens  in  front  of  the  global 
situation  display.  Such  a  lens,  when  placed  over  a  standard  video  monitor  or  diffusing  screen,  creates  an 
expanded  (by  about  50%)  and  optically  distant  image  that  can  easily  depict  a  180-degree  view  of  the  world  within 
its  boundary.  In  general,  global  situation  displays  should  be  placed  at  the  highest  point  of  the  control-station, 
which  in  an  aircraft  would  be  near  “head-level,”  just  beneath  the  cockpit  shroud. 

Because  the  auditory  system  provides  an  important  sensory  input  to  the  action  extrapersonal  neural 
system,  the  recent  use  of  3-D  auditory  cueing  devices  to  orient  to  out-the-window  tactical  targets  (McKinley  et  al., 
1995)  is  commensurate  with  the  natural  role  of  auditory  inputs  in  altering  us  to  distant  sources.  However,  3-D 
auditory  cueing  may  be  of  less  value  in  a  confined  aircraft  control-station,  for  the  following  reasons:  I)  auditory 
cueing  devices  are  much  less  effective  inside  the  focal  extrapersonal  visual  realm  (e.g.,  less  than  30  degrees  of 
eccentricity)  (Barfield  et  al.,  1997),  2)  auditory  cues  are  no  more  salient  than  visual  orientation  cues  (e.g.,  pulsing 
of  targets)  in  eliciting  head  movements  at  eccentricities  less  than  40  deg  (Goossens  &  Van  Opstal,  1997),  and  3) 
the  spatial  resolution  of  the  auditory  system  (-1  degree)  is  about  50  times  poorer  than  that  of  the  visual  system 
inside  the  central  visual  field  (Perrott  et  al.,  1987). 

Finally,  the  action  extrapersonal  system  provides  important  predictive  navigational  information  concerning  our 
heading  in  space  to  the  rest  of  the  brain,  as  illustrated  by  the  anticipatory  role  of  head  movements  during  locomotion 
(Grasso  et  al.,  1996);  hence,  it  is  desirable  to  provide  predictive  aircraft  path  information  in  an  aircraft  control  display. 
Although  the  flight  path  marker  on  current  head-up  displays  provides  predictive  information  concerning  the  current 
trajectory  of  the  aircraft  in  earth-fixed  space,  flight  path  directors  are  also  required  in  topographical  space  for  navigational 
purposes.  The  most  well-known  of  such  directors  is  the  “highway-in-the-sky,”  which  reportedly  leads  to  improved 
situational  awareness  and  navigational  performance  relative  to  traditional  steering  bars  (Reising  et  al.,  1996). 

Ambienl  Exlrapersonal.  The  major  function  performed  by  our  ambient  extrapersonal  system  is  to  spatially 
orient  our  heads  and  bodies  within  a  coordinate  frame  defined  by  the  gravity  vector  and  the  horizontal  plane  of  the  earth. 
Spatial  orientational  information  is  used  for  postural  control  (involving  predominantly  the  lower  limbs)  and  to  perceptu¬ 
ally  stabilize  the  world  up  to  roll  angles  of  60  degrees,  which  then  greatly  simplifies  the  perceptual  operations  carried  out 
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by  the  other  systems  (Previc,  1998).  In  our  terrestrial  environment,  transient  vestibular  and  somatosensory  inputs  are 
complemented  by  more  sluggish  visual  inputs  in  achieving  effective  spatial  orientation,  but  the  two  nonvisual  systems  are 
rendered  unreliable  in  the  abnormal  acceleratory  environment  of  flight  by  virtue  of  the  fact  that  the  gravitational  force  that 
they  ordinarily  sense  is  replaced  by  a  resultant  gravitoinertial  force  vector  that  can  deviate  quite  substantially  from  the 
direction  of  true  gravity.  Hence,  pilots  tend  to  be  more  visually  field-dependent  than  novices  as  they  rely  on  the  most 
critical  sources  of  spatial  orientation  (attitude)  information  available  to  them-namely,  the  ground  plane  and  horizon.  It 
has  been  shown  in  numerous  studies  that  the  visual  stimulus  characteristics  that  are  most  critical  for  achieving  “visual 
dominance”  in  spatial  orientation  are  depth  (distant  images  are  more  powerful)  and  field-of-view  (generally,  stimulus 
motion  within  the  central  50  degrees  is  ineffective  in  achieving  visual  dominance)  (Previc  &  Neel,  1 995). 

Although  many  efforts  to  duplicate  the  ambient  visual  realm  have  been  made  in  aircraft-control  workstations- 
ranging  from  peripheral  visual  devices  to  wide-field-of-view  helmet-mounted  displays  (Previc  et  al.,  1 992) — it  has 
heretofore  proven  impossible  to  provide  the  full  complement  of  cues  that  allow  a  scene  to  be  truly  stabilized  perceptu¬ 
ally.  Thus,  it  has  been  argued  that  the  artificial  horizon  in  attitude  displays  should  not  move  in  concert  with  the  image  of 
the  actual  horizon  on  the  retina,  but  rather  it  should  be  perceptually  stabilized  like  the  actual  horizon  is  in  our  “psychic 
reflection”  of  3-D  space  (Previc  & 

Ercoline,  1998).  The  stabilized 
horizon  attitude  concept-also  known 
as  the  outside-in  attitude  indicator  ( 

Figure  4)  because  it  assumes  a  view 
that  lies  directly  behind  the  aircraft,  as 
is  consistent  with  the  way  we  view 
ourselves  in  space  (Parsons,  1 987) — is 
especially  desirable  in  controlling 
aircraft  remotely.  Regardless  of  which 
type  of  attitude  display  is  selected, 
however,  the  primacy  of  attitude 
awareness  in  aircraft  control  and 
operations  requires  that  it  be  present  at 
all  times  and  preferably  at  or  near  the 
center  of  the  display  console. 

Because  of  the  tremendous 
demands  on  visual  processing  in  the 
cockpit,  there  have  been  several 
attempts  to  use  nonvisual  sensory  systems  to  relay  attitude  information  to  pilots.  One  effort  has  been  to  use 
auditory  cues  to  provide  primary  flight  information  to  the  pilot,  much  as  older  aircraft  did  by  virtue  of  changes  in 
the  amplitude  and  frequency  of  airframe  noise  with  changes  in  airspeed  and  attitude.  While  partly  beneficial  in 
the  absence  of  visual  cues  (Lyons  et  al ,  1990),  auditory  orientation  devices  are  hindered  by  the  fact  that  ambient 
auditory  cues  do  not  normally  contribute  greatly  to  maintaining  spatial  orientation  on  earth  (Lackner,  1977).  A 
more  recent  effort  has  been  to  use  tactile  information  to  convey  changes  in  pitch  and  bank  attitude  to  the  pilot 
(Rupert  et  al.,  1996).  While  the  tactile  sense  is  one  that  is  naturally  used  in  spatial  orientation  (  Table  1),  the  only 
tactile  orientation  vest  that  has  heretofore  been  prototyped  does  not  actually  attempt  to  recreate  the  natural 
stimulation  patterns  relayed  by  the  somatosensory  system  with  changes  in  orientation  relative  to  gravity  and  so 
may  not  create  a  truly  ambient  (preconscious)  orientational  percept. 

From  the  standpoint  of  motor  control,  the  major  guideline  would  be  to  use  the  primary  system  involved  in 
postural  control  and  locomotion  on  earth--the  lower  limbs  (feet).  Although  the  data  are  not  extensive,  foot-control  is 
very  effective  in  the  control  of  displays  and  is  comparable  to  hand-control  in  this  respect  (Kroemer,  1969).  Historically, 
the  feet  have  been  used  to  control  aircraft  yaw  by  means  of  the  rudder  pedals,  but  this  is  no  longer  necessary  with  fly-by- 
wire  control  systems.  Advanced  force-sensitive  foot  pedals  could  either  be  used  to  control  actual  attitude  in  space  (as  is 
done  by  skiers)  or  to  control  speed  (as  is  done  in  automobiles,  sewing  machines,  etc.).  Yaw  control  could  then  be  easily 
integrated  with  bank  movements  in  a  single  control-stick.  This  would  free  up  at  least  one  hand  at  a  given  moment  for 
control  operations  of  a  peripersonal  nature  (e.g.,  cursor  movements  on  a  screen). 

Finally,  the  neuropsychological  model  of  3-D  space  also  leads  to  predictions  concerning  how  orientational 
information  should  be  presented  in  helmet-mounted  displays  when  the  pilot  looks  away  from  the  longitudinal  axis  of  the 
aircraft.  Because  spatial  orientation  is  referenced  mainly  to  the  position  of  the  torso  and  legs  in  space  (Previc,  1998), 


Figure  4.  An  illustration  of  the  " moving- 
aircraft  ("outside-in")  attitude  display1  concept 
vs.  the  moving-horizon  ("inside-out")  concept. 
(From  Johnson  and  Roscoe,  1972). 
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head  movements  relative  to  the  body  should  not  alter  the  way  the  body  is  perceived  as  being  oriented  in  the  pilot’s  mental 
representation.  Thus,  attitude  information  should  arguably  be  less  slaved  to  where  the  pilot  is  looking  than  to  where  the 
aircraft  (and  the  pilot’s  torso)  are  pointed  in  space  (see  also  Worringham  &  Beringer,  1989).  Although  pilots  have  noted 
that  tradin’  nal  aircraft-referenced  inside-out  attitude  symbologies  -e  more  difficult  to  interprc  when  the  head  is  moved 
off-axis,  this  may  not  be  as  true  of  outside-in  and  other  displays  that  do  not  attempt  to  conform  to  the  retinal  projection  of 
the  outside  world. 

An  “Ideal”  Aircraft  Workstation  Based  on  Neuropsychological  Principles 

A  depiction  of  an  actual  aircraft  control-station  that  adheres  to  most  of  the  neuropsychological  guidelines 
put  forth  in  the  previous  section  is  shown  in  Figure  5.  It  features  a  sloping  console,  a  large  global  situation 
(topographical)  display  in  its  uppermost  portion,  an  outside-in  attitude  display  in  its  center,  dedicated  targeting 
and  “God’s-eye”  topographical  map  displays  just  below,  an  aircraft  status  display  located  in  the  lower  portion  of 
the  console,  a  single  control-stick  surrounded  by  panels  that  fall  within  normal  reaching  space,  and  foot  pedals  for 
at  least  airspeed  control.  This  control-station  could  be  transferred  to  almost  any  environment,  with  room  available 
for  additional  visual  displays  above  the  console,  and  it  could  be  easily  built  with  existing  technologies. 

At  first  glance,  the  “neuro- 
spychologically  compatible”  cockpit 
shown  in  Figure  5  may  not  appear  to 
represent  a  major  departure  from 
current  advanced  cockpit  stations 
which  already  incorporate  many  of 
its  key  features,  such  as  the  Boeing 
(McDonnell-Douglas)  Advanced 
Technology  Crew  Station  Nor 
would  such  a  control  station 
eliminate  all  of  the  task-saturation 
problems  facing  pilots  of  single-seat 
fighter  aircraft.  With  minimal 
training,  however,  pilots  with  this 
cockpit  console  would  arguably 
experience  enhanced  situational 
awareness,  greater  motoric  flexibility,  reduced  cognitive  stress,  and  an  major  overall  performance  edge  relative  to 
those  flying  with  current  cockpit  designs.  The  ability  to  incorporate  large  amounts  of  electronic  flight  informa¬ 
tion  into  control-stations  that  can  be  managed  with  great  ease  and  minimal  training  could  also  revolutionize  the 
general  aviation  world,  and  could  enable  the  safe  and  effective  control  of  uninhabited  combat  aerial  vehicles  to  be 
conducted  from  mobile  platforms  using  “nonpilot”  operators. 
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Executive  Summary 

The  Tactile  Situation  Awareness  System  (TSAS)  is  a  display  that  uses  the  under-utilized  sensory  channel 
of  touch  to  provide  spatial  awareness  information  to  pilots.  The  TSAS  system  accepts  data  from  various 
sensors  and  displays  this  information  via  vibrators  or  tactors  integrated  into  flight  gear.  Using  TSAS, 
pilots  have  demonstrated  improved  control  of  aircraft  during  complex  flight  conditions.  The  tactile 
display  has  been  shown  to  increase  Situation  Awareness  (SA)  and  provide  the  opportunity  to  devote  more 
time  to  other  instruments  and  systems  when  flying  in  task  saturated  conditions.  The  TSAS  system 
reduced  pilot  workload  and  thus  has  the  potential  to  increase  mission  effectiveness. 

TSAS  has  the  capability  of  providing  a  wide  variety  of  flight  parameter  information;  attitude,  altitude,  velocity, 
navigation,  acceleration,  threat  location,  targets,  etc.  However,  it  was  found  that  presenting  two  or  more 
different  types  of  information  simultaneously  and  in  close  proximity  makes  the  system  non-intuitive  and 
difficult  to  use.  This  is  due  in  part  to  limitations  of  current  tactor  technology.  This  problem  of  "information 
overload"  is  also  seen  in  visual  and  audio  displays.  The  concept  of  intelligent  software  that  presents  the  most 
critical  piece  of  information  when  needed  is  under  development.  For  example,  during  a  night  carrier  launch, 
angle  of  attack  is  of  prime  concern,  at  cruise  altitude,  navigation  information  is  important  until  perhaps  threat  or 
target  information  is  required  When  and  how  to  transition  from  one  mode  to  another  is  a  current  software 
development  effort  of  the  TSAS  project.  The  software  system  allows  different  types  of  information  to  be 
displayed  through  automatic,  rule-based  mode  switching. 

TSAS,  integrated  with  visual  and  audio  display  systems  represents  the  basis  for  the  next-generation  human 
systems  interface  for  the  tactical  aircraft  of  tomorrow.  Mode  switching  software  development  will  facilitate  the 
eventual  integration  of  visual,  audio,  and  tactile  displays  into  a  synergistic  spatial  awareness  display  that  will 
provide  the  right  information  at  the  right  time  by  the  right  sensory  channel(s). 

Introduction 

Spatial  disorientation  (SD)  mishaps  have  occurred  ever  since  man,  who  evolved  in  an  essentially  two- 
dimensional  terrestrial  environment,  entered  the  three-dimensional  aerospace  environment.  In  our  day-to-day 
terrestrial  activities,  spatial  orientation  is  continuously  maintained  by  accurate  information  from  three  independ¬ 
ent,  redundant,  and  concordant  sensory  systems:  vision,  the  vestibular  system  or  inner  ear,  and  the  somatosen¬ 
sory  system  (skin,  joint,  and  muscle  sensors).  These  complementary  and  reliable  sources  of  information  are 
integrated  in  the  central  nervous  system  to  formulate  the  appropriate  perception  of  orientation  and  motor 
response. 

However,  in  aerospace  environments,  the  vestibular  and  somatosensory  (tactile)  sensations  no  longer  provide 
accurate  information  concerning  the  magnitude  or  direction  of  the  gravity  vector.  For  example,  in  the  military 
aviation  environment,  the  almost  continuous  changes  in  acceleration  and  direction  of  aircraft  motion  expose 
aircrew  to  a  resultant  gravito-inertial  force  that  is  constantly  changing  in  magnitude  and  direction.  Under  such 
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circumstances,  somatosensory  and  vestibular  information  concerning  the  direction  of  "down”  may  be  incorrect, 
and  increased  reliance  must  be  placed  on  visual  information 

As  long  as  aviators  can  maintain  visual  reference  with  respect  to  the  ground  or  horizon,  spatial  orientation  does 
not  pose  a  significant  problem.  However,  in  early  aviation,  ’’cloud  flying”  and  other  forms  of  flight  in  reduced 
visibility  claimed  the  lives  of  aviators  w  ith  an  alarming  frequency.  The  incidence  of  SD  declined  significantly 
when  Sperry  introduced  the  gyro-stabilized  artificial  horizon.  However,  SD  mishaps  were  not  eliminated 
completely  because  the  gyro-stabilized  artificial  horizon  or  attitude  indicator  is  a  visual  instrument  and  only 
provides  orientation  information  when  the  aviator  is  looking  at  the  instrument. 

Today,  the  typical  SD  mishap  occurs  when  the  visual  system  is  compromised  (e.g.,  temporary  distraction, 
increased  workload,  transitions  between  visual  and  meteorological  conditions,  or  reduced  visibility).  The 
central  nervous  system  must  then  compute  spatial  orientation  with  the  remaining  information,  vestibular  and 
somatosensory,  at  its  disposal,  which  is  frequently  incorrect  In  fact,  it  is  a  physiologically  normal  response  to 
experience  SD  in  such  circumstances.  The  opportunities  for  SD  mishaps  are  constantly  increasing  due  to  more 
frequent  night  operations,  requirements  for  all  weather  fly  ing,  increased  low-level  or  ”nap  of  the  earth”  flight, 
the  use  of  night  vision  goggles  (which  decreases  peripheral  vision)  and  improved  agility  of  aircraft.  In  addition, 
a  more  demanding  pilot  workload  produces  fatigue  during  sustained  operations.  All  of  these  factors  are 
conducive  to  SD. 

When  aircraft  mishaps  are  categorized  by  causation  factors,  the  largest  single  factor  is  consistently  pilot  error. 
The  U.S.  Air  Force  has  indicated  that  the  most  significant  human-factors  problem  is  SD.  In  the  aviation 
environment,  SD  occurs  when  pilots  incorrectly  perceive  the  attitude,  altitude,  or  motion  of  their  aircraft  relative 
to  the  earth  or  other  significant  objects.  SD  is  a  triservice  aviation  problem  that  annually  costs  DoD  in  excess  of 
S300  million  in  lost  aircraft.  Of  the  15  Navy  aircraft  lost  to  noncombatant  action  in  the  Desert  Shield/Storm 
conflict,  7  were  SD  mishaps. 

In  an  effort  to  provide  the  aviator  with  orientation  and  other  mission  critical  information,  sophisticated  visual 
displays  including  Multi-Function  Displays  (MFDs)  and  Head  Mounted  Displays  (HMDs)  have  been  devel¬ 
oped.  MFDs  and  HMDs  are  capable  of  displaying  large  amounts  of  information  by  using  different  modes  or 
“pages”.  Each  display  page  contains  specific  information  (engine,  hover  mode,  horizontal  situation  or  vertical 
situation,  etc.).  Switching  between  pages  to  obtain  the  required  information  is  either  automatic  or  pilot 
selectable.  Just  as  in  Sperry's  time,  these  sophisticated  displays  provide  no  information  when  the  visual  system 
is  compromised.  Why  is  it  that  since  Sperry'  there  have  been  no  further  landmark  developments  by  the  human- 
factors  engineers  in  introducing  displays  or  instrumentation  to  solve  orientation  problems? 

Studies  by  the  U.S.  Army  (Simmons,  et  al.  1  D78a,b)  indicate  that  pilots  in  instrument  flight  conditions  spend 
more  than  50  percent  of  their  visual  scan  time  attending  to  two  instruments,  the  attitude  indicator  and  the 
directional  gyro  (ADI).  By  presenting  this  information  non-visually,  pilots  will  be  free  to  attend  to  other  tasks 
and  instruments  that  do  require  vision  attention 

The  Tactile  Situation  Awareness  System  (TSAS)  is  a  non-visual  display  that  uses  the  under-utilized  sensory 
channel  of  touch  to  provide  orientation  information  to  pilots  (Rupert  et  al.  1989,  1993).  The  approach,  as  shown 
in  Figure  1,  is  to  use  a  tactor  locator  system  fitted  with  multiple  vibrating  tactors  that  can  continuously  update 
the  pilot’s  awareness  of  position-analogous  to  how  our  brain  obtains  orientation  information  in  the  terrestrial 
environment.  Thus  the  pilot  will  be  able  to  maintain  orientation  in  the  absence  of  a  visual  horizon  or  during 
inevitable  temporary'  gaze  shifts  from  the  aircraft  instrument  panel 
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Figure  1:  TSAS  System  Configuration 

TSAS  has  the  capability  of  providing  a  wide  variety  of  flight  parameter  information,  such  as  attitude,  altitude, 
velocity,  navigation,  threat  location,  targets,  etc.  However,  we  have  found  that  presenting  two  or  more  different 
types  of  information  simultaneously  and  in  close  proximity  makes  the  system  non-intuitive  and  difficult  to  use. 
This  is  due  in  part  to  limitations  of  current  factor  technology.  This  problem  of  “information  overload"  is  also 
seen  in  visual  and  audio  displays. 

The  concept  of  developing  intelligent  software  that  presents  the  most  critical  piece  of  information  when  needed 
is  under  development.  This  is  similar  to  the  “page”  concept  of  state-of-the-art  visual  displays.  A  page  presents 
certain  specific  information  on  the  visual  display  (about  engines,  aircraft  attitude,  radar,  etc.).  For  example, 
during  a  helicopter  instrument  take-off,  pitch  attitude,  positive  rate  of  climb,  and  heading  are  of  prime  concern, 
whereas  at  cruise  altitude,  navigation  information  is  important.  When  and  how  to  transition  from  one  mode  to 
another  is  a  current  software  development  effort  of  the  TSAS  project.  The  software  system  allows  different 
types  of  information  to  be  displayed  through  automatic,  rule-based  mode  switching 

TSAS,  integrated  with  HMDs  and  3D  audio  systems,  represents  the  basis  for  the  next-generation  human 
systems  interface  for  the  tactical  aircraft  of  tomorrow.  Development  of  mode-switching  software  mechanisms 
for  the  tactile  display  will  also  be  applicable  to  advanced  HMD  and  3D  audio  displays.  Such  software  devel¬ 
opment  will  facilitate  the  eventual  integration  of  visual,  audio,  and  tactile  displays  into  a  synergistic  display  that 
provides  the  right  information  at  the  right  time  by  the  right  sensory  channel(s). 

TSAS  Flight  Demonstration  Review 

T-34:  NAWCAD,  NAS  Patuxent  River,  MD.  October  1995. 

The  T-34  Tactile  Situation  Awareness  System  (TSAS)  flight  demonstration  project  (Rupert  et  al.  1996)  was 
originated  at  the  NASA  Johnson  Space  Center  and  the  Naval  Aerospace  Medical  Research  Laboratory  with 
funding  provided  by  the  Office  of  Naval  Research  (ONR)  through  the  Advanced  Technology  Demonstration 
(ATD)  program.  The  T-34  TSAS  project  integrated  a  tactile  display  into  a  T-34  aircraft  (Raj  et  al.  1996).  A  7- 
event  test  operation  was  conducted  to  demonstrate  that  a  pilot  could  maintain  aircraft  orientation  using  a  tactile 
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display  during  flying  operations.  One  Unites  States  Navy  (USN)  test  pilot  was  selected  to  fly  the  flight  events. 
Objectives  of  the  T-34  TSAS  flight  demonstration  program  were  to  demonstrate: 

•  That  a  significant  amount  of  orientation  and  awareness  information  can  be  intuitively  provided 
continuously  by  the  under-utilized  sense  of  touch 

•  The  use  of  the  TSAS  display  to  show  that  a  pilot,  with  no  visual  cues,  can  effectively  maintain 
control  of  the  aircraft  in  normal  and  acrobatic  flight  conditions. 


Figure  2:  TSAS  equipped  T-34 

The  first  flight  of  the  TSAS-modified  T-34  (Figure  2)  was  1 1  October  1995,  and  7  flight  test  events  were 
successfully  completed  by  19  October  1995.  Summary'  results  showed  that  veridical  roll  and  pitch  tactile  cues 
could  be  provided  via  a  matrix  array  of  vibrotactors  incorporated  into  a  Tactor  Locator  System  (TLS).  The  TLS 
was  a  torso  harness  of  cotton  and  fire  retardant  NOMEX  with  elastic  and  velcro  straps  that  was  worn  under¬ 
neath  the  flight  suit  The  TLS  positioned  the  20  electro-mechanical  tactors  on  the  torso  of  the  body  as  an  array 
that  consisted  of  four  columns  of  five  tactors  located  90  degrees  apart  on  the  front,  left,  back,  and  right  of  the 
test  pilot.  The  test  pilot  in  the  rear  seat  was  shrouded  to  block  any  outside  visual  cues  and  all  flight  instruments 
in  the  rear  cockpit  were  removed  The  test  pilot  flew  the  following  maneuvers  straight  and  level  for  5  minutes; 
climbing  and  descending  turns;  unusual  attitude  recovery;  loops;  aileron  rolls;  and  ground  controlled  approaches 
(GCA).  The  test  pilot  successfully  performed  all  maneuvers  without  visual  cues,  relying  solely  on  tactile  cues 
for  attitude  information. 

For  this  flight  demonstration,  two  different  “tactile  pages”  were  developed  using  variations  of  tactor  stimulus 
selected  from  intensity,  and  location.  The  first  tactor  page  was  used  for  fine  control  of  the  aircraft,  and  the 
second  program  was  used  for  acrobatic  control.  The  first  or  “fine”  page  had  a  pitch  and  roll  range  of  +/-  40 
degrees  and  used  only  three  tactors  per  column,  and  the  second  or  “acrobatic”  page  had  a  pitch  and  roll  range  of 
+/-  180  degrees  and  used  five  tactors  per  column.  Flight  performance  showed  that  it  was  more  intuitive  and 
easier  to  distinguish  between  a  low,  middle  and  high  tactor,  therefore  during  non-acrobatic  maneuvers  improved 
performance  was  noted  using  the  fine  page  as  compared  to  using  the  five  tactor  acrobatic  page.  Only  the 
acrobatic  control  program  could  be  used  for  the  acrobatic  maneuvers.  This  result  indicates  that  the  tactile 
display  should  provide  information  optimized  for  a  particular  flight  regime,  and  that  the  test  pilot  could 
distinguish  between  tactor  "pages." 

UH-60  USAARL,  FT  Rucker,  AL.  December  1995 


The  UH-60  TSAS  flight  demonstration  project  was  a  follow-on  effort  to  the  fixed-wing  T-34  TSAS  flight 
demonstration  and  integrated  the  tactile  display  into  an  UH-60  helicopter.  A  9-event  test  operation  was 
conducted  to  demonstrate  that  a  rotary  wing  pilot  could  receive  aircraft  orientation  and  performance  information 
using  a  tactile  display  during  flying  operations.  Three  US  Army  (USA)  pilots  were  selected  to  fly  the  flight 
events.  Objectives  of  the  UH-60  TSAS  flight  demonstration  program  were  to  demonstrate- 

•  That  a  significant  amount  of  orientation  and  awareness  information  can  be  intuitively  provided 
continuously  by  the  under-utilized  sense  of  touch 
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•  The  use  of  the  TSAS  display  to  show  that  a  pilot,  with  no  visual  cues,  can  effectively  maintain 
control  of  a  helicopter  in  the  complex  rotary  wing  environment 

Hie  first  flight  of  the  TSAS-modified  UH-60  was  1 1  December  1995,  and  9  flight  test  events  were  successfully 
completed  by  20  December  1995.  The  flight  demonstrations  showed  that  controlled  flight  maneuvers  in  a 
rotary'  wing  aircraft  using  tactile  information  was  possible.  Roll  and  pitch  ♦actile  cues  were  provided  via  a 
matrix  array  of  vibrotactors  incorporated  into  a  torso  harness  as  used  for  the  T-34  effort.  Additionally,  airspeed 
and  heading  error  tactile  cues  were  provided  by  tactors  located  on  the  arms  and  legs,  respectively.  The 
“blindfolded”  test  pilot  in  the  right  seat  had  no  visual  cues.  The  test  pilots  flew  the  following  maneuvers, 
straight  and  level,  standard  rate  turns;  unusual  attitude  recovery  and  GCA.  The  test  pilots  were  able  to  success¬ 
fully  perform  all  maneuvers  without  visual  cues,  relying  solely  on  tactile  cues  for  the  necessary  attitude  and 
performance  information. 

The  auxiliary  tactile  cues  off  the  torso  caused  some  difficulties.  Heading  control  remained  problematic  during 
the  demonstration,  pilots  had  some  difficulty  picking  up  the  heading  error  signal  when  the  other  pitch  and  roll 
channels  were  active.  This  suggests  that  keeping  the  heading  tactors  off,  until  the  pilot  approached  straight  and 
level  flight  would  improve  the  ability  to  return  to  a  base  course  following  unusual  attitude  recovery. 

JSF  /  UH-60:  NAMRL/USAARL,  NAS  Pensacola,  September  1 997 

The  Joint  Strike  Fighter  (JSF)  -Tactile  Situation  Awareness  System  (TSAS)  flight  demonstration  project  (Rupert 
and  McGrath,  1998)  was  a  short-duration  technology  maturation  and  flight  demonstration  program  funded  by 
the  JSF  Program  Office.  The  JSF-TSAS  project  integrated  a  tactile  display,  F-22  cooling  vest,  and  global 
positioning  system/inertial  navigation  system  (GPS/INS)  technologies  into  an  UH-60  Helicopter  (Figure  3).  A 
10-event  test  operation  was  conducted  to  demonstrate  the  utility  of  this  advanced  human-machine  interface  for 
performing  hover  operations.  Four  test  pilots  were  selected  to  fly  the  flight  events. 


Figure  3  TSAS  equipped  UH-60 

Objectives  of  the  JSF-TSAS  flight  demonstration  program  were  to  demonstrate: 

•  TTie  potential  for  TSAS  technology  to  reduce  pilot  workload  and  enhance  Situation  Awareness 
(SA)  during  hover  and  transition  to  forward  flight. 

•  The  use  of  the  TSAS  display  to  show  that  pilots,  with  no  outside  visual  cues,  can  effectively 
hover  and  transition  to  forward  flight  in  a  vertical  lift  aircraft. 

The  first  flight  of  the  TSAS-modified  UH-60  was  09  September  1997,  and  10  flight  test  events  were  success¬ 
fully  completed  by  1 9  September  1997. 

Summary  results  showed  that  TSAS  technologies  increase  pilot  SA  and  reduce  pilot  workload  when  using  the 
tactile  display,  especially  during  simulated  shipboard  operations  in  Instrument  Meteorological  Conditions 
(1MC).  Prototype  hardware  development  showed  that  tactile  displays  could  be  integrated  into  existing  flight 
gear  (Figure  4).  The  test  pilots  successfully  performed  all  maneuvers  with  degraded  outside  visual  cues,  relying 
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on  tactile  cues  for  the  necessary  information.  Using  TSAS,  pilots  demonstrated  improved  control  of  aircraft 
during  complex  flight  conditions.  The  tactile  display  reduced  pilot  workload  and  provided  the  opportunity  to 
devote  more  time  to  other  instruments  and  systems  when  flying  in  task  saturated  conditions.  These  effects  can 
potentially  increase  mission  effectiveness.  One  USA  test  pilot  commented  that  “in  1MC,  the  TSAS  vest  could 
be  the  difference  between  mission  success  and  a  mishap.” 

The  awareness  of  aircraft  velocity  over  the  ground  or  “drift”  without  looking  at  a  visual  instrument  was  the 
biggest  advantage  of  the  JSF-TSAS.  The  maintenance  of  SA  during  hovering  in  reduced  visual  conditions  was 
enhanced.  Overall,  TSAS  decreased  pilot  workload  and  enhanced  SA. 


Figure  4:  TSAS  Experiment  Pilot  Showing  TSAS  factor  Locator  System 


In  conclusion,  the  three  TSAS  flight  demonstrations  have  shown  that  a  pilot  can  maintain  control  of  an  aircraft 
using  tactile  cues,  and  that  a  tactile  display  can  reduce  workload  and  enhance  SA.  However,  development  of 
the  following  issues  is  necessary: 

1 .  The  need  for  improved  tactors  The  prototype  tactors  used  in  the  flight  demonstrations  could  only  be 
tunned  on  and  off.  The  amplitude,  frequency,  and  stimulus  type  (vibration,  stroking)  could  not  be 
controlled  in  the  prototype  system.  This  is  analogous  to  a  black  and  white  vs.  a  color  visual  display. 
A  “richer”  tactile  sensation  that  could  convey  multiple  information  can  be  achieved  with  improved 
tactors. 

2.  Incorporating  the  absolute  minimum  number  of  tactors  into  existing  flight  gear.  Today’s  aviator  is 
asked  to  wear  an  ever-increasing  amount  of  equipment.  A  tactile  display  that  has  a  minimum  number 
of  tactors  that  still  provides  the  necessary  information  will  be  lighter,  more  robust,  and  easier  to  main¬ 
tain  than  a  tactile  display  with  a  large  number  of  tactors 

3.  Keeping  the  tactile  display  intuitive  and  easy  to  understand. 

To  achieve  goals  2  and  3,  the  concept  of  developing  intelligent  software  that  presents  the  most  critical  piece  of 
information  when  needed  is  required.  When  and  how  to  transition  from  one  mode  to  another  is  a  current 
software  development  effort  of  the  TSAS  project.  The  software  system  allows  different  types  of  information  to 
be  displayed  through  automatic,  rule-based  mode  switching. 

Intelligent  Knowledge-Based  Software 

Intelligent  knowledge-based  software  is  a  computer  program  that  enables  a  computer  to  make  a  decision  that  is 
normally  made  by  a  human  with  special  expertise.  This  is  also  termed  expert  system.  The  architecture  of 
intelligent  knowledge-based  software  is  somewhat  reminiscent  of  human  cognitive  structures  and  processes. 
The  first  part  of  human  expertise  is  a  long-term  memory  of  facts,  structures,  and  rules  that  represent  expert 
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knowledge  about  the  domain  of  expertise.  The  analogous  structure  in  intelligent  knowledge-based  software  is 
called  the  knowledge  base.  The  second  part  of  human  expertise  is  a  method  of  reasoning  that  can  use  the  expert 
knowledge  to  solve  problems.  The  part  of  intelligent  knowledge-based  software  that  carries  out  the  reasoning 
function  is  called  the  inference  engine.  In  this  analogy  the  inference  engine  mimics  thinking,  while  knowledge 
is  contained  in  the  knowledge  base. 


A  rule-based  inference  engines  centers  on  the  use  of  IF  THEN  statements.  For  example, 

1  If  the  helicopter  forward  airspeed  is  less  than  20  knots,  then  the  helicopter  is  hovering. 

2.  If  the  helicopter  is  hovering  and  the  helicopter  pitch  and  roll  attitude  is  less  than  15  degrees  then 
the  tactile  display  should  provide  hovering  information. 


When  the  current  problem  situation  satisfies  or  matches  the  IF  part  of  a  rule,  the  action  specified  by  the  THEN 
part  of  the  rule  is  performed,  Because  intelligent  knowledge-based  software  is  dealing  with  fast  moving  data, 
rules  offer  the  opportunity  to  examine  the  state  of  the  data  at  each  step  and  react  appropriately.  The  use  of  rules 
also  simplifies  the  job  of  explaining  what  the  program  did  or  how  it  reached  a  particular  conclusion. 


rhe  architecture  for  the  intelligent  knowledge-based  software  under  development  for  the  TSAS  project  is 
shown  in  Figure  5.  The  software  system  is  comprised  of  the  following  major  components: 

1 .  Input  modules  that  provide  the  system  with  actual  information  on  aircraft  state,  aircraft  perform¬ 
ance  data,  pilot  inputs,  and  pilot  procedures. 

2.  Model  modules  that  provide  theoretical  information  about  the  aircraft  and  pilot  to  the  knowledge 
base. 

3.  The  knowledge  base  interacts  with  the  inference  engine  and  organizes  and  stores  all  available  in¬ 
formation. 

4  Rule-based  inference  engine  which  interacts  with  the  input  modules,  model  modules  and  the 
knowledge  base  and  then  determines  the  right  information  to  be  displayed  in  a  particular  situa¬ 
tion. 
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Figure  5  TSAS  Intelligent  Knowledge-Based  Softw'are  Architecture 

Aircraft  Sensor  Module  is  an  input  module  to  the  knowledge  base  and  provides  the  actual  state  of  the  aircraft 
(air  speed,  attitude,  etc.)  and  the  actual  state  of  aircraft  systems  (engine  temperature,  rpm,  etc.). 
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Aircraft  Controls  Module  is  an  input  module  that  provides  the  actual  pilot  inputs  on  the  controls  (throttle,  cyclic, 
etc.). 

Aircraft  Data  Module  is  an  input  module  that  provides  aircraft  performance  data.  This  data  are  typically 
empirical  in  nature  and  is  supplied  by  the  aircraft  manufacturer. 

Human  Model  Module  is  a  predictive  module  that  contains  a  model  of  the  perceived  human  orientation.  The 
input  to  this  model  are  data  from  the  aircraft  sensor  module.  This  model  uses  observer  theory  techniques  to 
estimate  the  perceived  orientation  of  the  pilot.  The  observer  theory  model  of  human  orientation  was  described 
by  Oman  (1982),  and  subsequently  enhanced  by  Borah  et  al.  (1988),  and  Pommellet  (1990).  In  the  inference 
engine,  data  from  this  module,  estimated  pilot  perception  of  orientation,  is  compared  to  actual  pilot  and  aircraft 
orientation,  and  an  estimate  of  the  potential  for  SD  is  predicted  Tactor  activation  together  with  visual  and 
auditory  displays  is  modified  when  an  SD  situation  is  predicted.  One  possible  implementation  of  this  concept  is 
to  increase  the  magnitude  of  the  tactile  sensation  during  periods  of  high  probability  SD  situations. 

Aircraft  Model  Module  is  predictive  model  of  the  actual  aircraft.  The  input  to  this  model  is  data  from  the 
aircraft  control  module  This  model  uses  a  computer  motion  model  of  the  aircraft  to  estimate  the  state  of  the 
aircraft  In  the  inference  engine,  data  from  this  module,  estimated  aircraft  state,  is  compared  to  actual  aircraft 
state  and  an  estimate  of  a  potential  aircraft  sensor  failure  or  unusual  meteorological  condition  is  predicted. 

The  Knowledge  Base  contains  "expert  knowledge"  of  aircrew  procedures  in  different  situations.  For 
example,  an  instrument  takeoff  in  a  SH60  requires  the  following  procedure  (Naval  Air  Training  and 
Operating  Procedures  Standardization  -  NATOPS  -  13.1.1 .402.c): 

At  50  KIAS,  as  the  Automatic  Flight  Control  System  (AFCS)  switches  to  airspeed  hold,  level 
the  wings,  place  feet  on  the  pedals  and  center  the  ball  if  required.  Accelerate  to  100  KIAS  and 
establish  a  minimum  of  500  FPM  Rate  of  Climb  (ROC).  Take  up  a  heading  to  account  for  drift 

Information  like  this  is  provided  to  the  rule-based  inference  engine  by  the  knowledge  base.  The  knowl¬ 
edge  base  organizes  and  stores  all  available  information. 


The  Rule- Based  Inference  Engine  determines  the  right  information  to  be  displayed  in  a  particular  situation.  As 
described  earlier,  the  rule-based  inference  engine  makes  extensive  use  of  IF  TF1EN  statements.  When  the 
current  problem  situation  satisfies  or  matches  the  IF  part  of  a  rule,  the  action  specified  by  the  THEN  part  of  the 
rule  is  performed. 

The  intelligent  knowledge-based  software  described  above  controls  the  tactile  presentation  of  information,  and 
this  software  must  be  adaptive,  and  "smart"  about  which  information  to  provide,  and  how,  when,  what,  where, 
to  provide  that  information 

How  to  provide  tactile  information? 

The  presentation  of  tactile  information  should  be  intuitive  and  easy  to  interpret.  It  should  be  neither  annoying 
nor  something  that  the  pilot  completely  habituates  to.  Additionally,  information  concerning  threats  or  warnings 
must  be  clearly  and  easily  differentiated  from  routine  information 

When  to  provide  tactile  information? 

The  software  system  must  monitor  a  variety'  of  flight  parameters,  pilot  inputs,  and  model  estimates  and  prioritize 
the  information  depending  on  the  current  context.  Some  examples  of  "when"  to  provide  tactile  information 
may  include: 

•  continuously  provide  orientation  information  to  help  the  pilot  maintain  spatial  awareness 

•  when  aircraft  is  in  autopilot  mode  and  during  transition  between  autopilot  and  pilot  control 

•  when  there  is  a  threat  from  hostile  aircraft 

•  when  the  input  and  predictive  modules  indicate  that  the  aircraft  might  soon  be  at  risk  or  that  the 
pilot  may  be  experiencing  SD. 
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What  information  to  provide? 

There  is  a  wide  range  of  information  that  can  be  provided  via  the  sense  of  touch,  howev  er,  flight  demonstrations 
have  shown  that  pilots  using  the  current  limited  prototype  display  can  only  usefully  accommodate  a  subset  of 
this  total.  Some  examples  of  "what”  information  to  provide  the  pilot  via  tactile  input  might  include: 

•  roll  and  pitch  information 

•  helicopter  or  VSTOL  aircraft  drift  information 

•  navigation  information 

•  relative  location  of  hostile  aircraft 

•  SD  episode  recovery  information 

•  instrument  landing  information 

Where  to  provide  tactile  information? 

The  tactors  must  be  located  to  achieve  the  goal  of  intuitive  and  easy  to  interpret  tactile  information.  Some 
examples  of  "where"  to  provide  information  might  include 

•  attitude  information  during  acrobatic  and  normal  flight  situations. 

Intelligent  knowledge-based  software  will  be  an  essential  and  complex  component  of  a  tactile  display 
system  that  provides  critical  information  to  the  pilot  in  a  non-visual  manner 

Conclusion 

TSAS  technologies  have  shown  that  tactile  displays  can  play  an  important  role  in  improving  situation  awareness 
in  modem  military  aircraft.  However,  to  achieve  a  complete  solution  to  the  problem  of  SD  and  subsequent  loss 
of  situation  awareness,  tactile  displays  must  be  integrated  with  helmet-mounted  displays  (HMDs)  and  3D  audio 
systems.  Intelligent  knowledge-based  software  that  uses  mode-switching  software  mechanisms  developed  for 
the  tactile  display  will  facilitate  the  eventual  integration  of  visual,  audio,  and  tactile  displays  into  a  single 
synergistic  spatial  awareness  display.  The  switching  software  must  be  adaptive,  and  "smart"  about  which 
information  to  provide,  and  how,  when,  what,  where,  to  provide  that  information.  The  spatial  awareness  display 
will  provide  the  right  combination  of  information  at  the  right  time  by  the  right  sensory  channel(s). 
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Controlled  Flight  into  Terrain  Hazards 
Associated  with  Army  Rotary  Wing  Aircraft 


Russell  Peusch 

US.  Army  Safety'  Center 

Ft.  Rucker,  AL 

Background.  This  study  was  initiated  to  provide  information  on  Army  accidents  to  help  with  a  decision  to  pursue 
installation  of  Ground  Proximity  Warning  Systems  (GPWS)  on  Army  rotary  w  ing  aircraft.  The  G  WPS  was  touted  to  be 
an  effective  technology  to  reduce  incidence  of  Controlled  Flight  into  Terrain  (CFIT). 

Vision  In  keeping  with  the  focus  of  the  Army  Safety  Program,  this  study  sought  to  support  the  Army  chain  of 
command  w  ith  risk  management  information.  Specifically,  this  study  was  to  provide;  (1 )  a  list  of  hazards,  (2)  the  cost 
associated  with  accidents  caused  by  these  hazards,  and  (3)  the  projected  impact  that  GPWS  could  provide  given 
previous  accidents  It  is  expected  that  the  Army  will  base  an  acquisition  decision  on  the  safety  risk  reduction  that  a 
given  technology  provides  weighed  against  cost  and  availability.  Thus,  this  study  provides  a  framework  to  compare 
GPWS  with  the  potential  effectivess  of  other  technologies. 

Methodology.  Several  challenges  needed  to  be  overcome  during  this  study.  The  first  challenge  was  the  lack  of  an 
official  DoD  definition  for  CFIT.  The  other  challenges  were  that  the  ASMIS  neither  categorized  CFIT  accidents  nor 
readily  identify  hazards  associated  with  accidents  With  an  assumed  definition  for  CFIT,  the  ASMIS  was  queried  for 
accident  profiles  that  fit  the  CFIT  definition.  Digital  data  concerning  these  accidents  were  reviewed  and  a  list  of  hazards 
was  developed  and  continuously  improved  during  the  review  process.  The  list  of  hazards  was  developed  using  the  US 
Army  Safety  Center  (USASC)  model  for  hazard  identification. 

Results.  Out  of  249  class  A  &  B  accidents  from  FY88  through  FY97  were  examined,  137  (55%)  were  determined  to  be 
CFIT  accidents.  From  these  accidents,  15  hazards  were  identified  among  3  families:  (I)  loss  of  visual  cues,  (2)  crew 
workload/task  saturation,  (3)  other  maneuvering  profiles. 

Conclusions.  A  high  payoff  for  the  Army  would  be  to  develop  short  term  (operational  user  and  commander),  medium 
term  (doctrine,  training,  standards,  and  combat  developer),  and  long  term  (materiel  developer)  solutions  to  reduce 
accidents  related  to  hazards  in  the  following  two  areas.  First,  loss  of  visual  cues  resulting  in  hover  drift  specifically  for 
the  AH-64A  and  the  OH-58D  aircraft.  Second,  workload  and  loss  of  visual  cues  hazards  where,  generally,  the 
aircrew’s  ability  to  judge  obstacle  clearance  or  other  altitude  requirements  during  low  level  flight,  and  specifically,  for 
the  AH-64A,  UH- 1  (all)  and  OH-58A/C.  These  two  areas  made  up  20%  of  all  aviation  Class  A  and  B  accidents  from 
FY93  through  FY97. 

Recommendations.  Based  on  the  conclusion  of  this  study,  the  following  are  submitted  as  recommendations:  (1)  the 
USASC  begin  work  to  articulate  the  risks  associated  with  the  hazards  identified  in  this  study,  (2)  the  USASC  oversee 
the  risk  management  of  these  hazards  throughout  the  Army  using  the  Chief  of  Staff  Army  (CSA)  Safety  1  PR  as  a 
medium  to  begin  the  process,  (3)  the  Director  of  Army  Safety  propose  objectives  for  reductions  in  specific  hazards  to 
the  CSA  to  implementing  a  performance  based  safety  program,  (4)  the  USASC  develop  a  methodology  to  articulate 
future  CFIT  accidents  as  an  expected  loss  rate  given  exposure  of  known  hazards  and  currently  existing  controls,  (5)  the 
USASC  apply  the  methodology  used  in  this  study  to  develop  hazards  associated  with  other  aviation  accidents  (e.g., 
multiple  aircraft  accidents/rmdair  collisions,  materiel  failures,  etc)  and  begin  to  apply  the  same  methodology  to  a  family 
of  ground  systems. 


(Reprint  of  executive  summary;  formal  paper  not  available.) 
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Practical  Considerations  for  Measuring  Situational  Awareness 


Steven  Adams 

Naval  Air  Warfare  Center  W^pons  Division 
Targeting  Systems  Integration  Section  (452330D) 
W  China  Lake,  CA  93555-6001 
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I.  Issues/Rationale  for  Development  of  an  In-Flight  SA  Scale 

Situational  Awareness  (SA)  subjective  metrics  that  are  commonly  available  are  in  many  cases  too  intrusive  to 
use  during  flight  test.  The  China  Lake  Situational  Awareness  (CLSA)  features  a  method  to  gather  Situational 
Awareness  data  during  flight  test  on  a  pass  by  pass  basis  to  permit  execution  of  multiple  test  events  per  flight.  Pilot 
kneeboard  eards  and  voice  annotated  Head  Up  Display  (HUD)  tapes  are  used  to  gather  pilot  subjective  SA  ratings  in 
the  cockpit  as  flight  test/demonstration  profiles  are  flown.  CLSA  is  administered  similar  to  the  Bedford  Workload 
Scale  developed  by  DRA  Famborough.  The  Bedford  Scale  (Figure  I )  is  used  as  a  Workload  measure  in  flight  since  it 
is  unidimensional  and  has  a  clearly  defined  set  of  criteria  for  ratings. 


Figure  1.  Bedford  Workload  Scale. 


Similarly,  CLSA  (Figure  2)  uses  a  five  point,  '‘Bedford-like”  SA  scale  that  was  abstracted  from  the  Situational 
Awareness  Global  Assessment  Technique  (SAGAT)  model  of  I)  environmental  status  and  trends,  2)  mission  status  and 
trends,  and  3)  awareness  of  what  will  happen  if  present  status/trends  continue  (and  needed  interventions  are  made  or 
omitted).  However,  SAGAT  is  objective  and  is  simulator  based,  whereas  CLSA  is  subjective  and  is  intended  for  flight 
test.  Other  flight  test/demonstration  resources  to  be  used  in  the  protocol  for  SA  measurement  with  CLSA  include  the 
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Bedford  Scale  itself,  trainer  aircraft  if  available  (for  the  two-seat  architecture),  a  safety  pilot  (if  available),  cockpit 
cameras,  real  time  data  downlinking,  hot  mic,  VHS  flight  tapes,  and  extensive  debriefs/questionnaires. 


SA  SCALE  VALUE 

CONTENT 

VERY  GOOD 

•  FULL  KNOWLEDGE  OF  A/C  ENERGY  STATE/TACTICAL 

ENVIRONMENT/MISSION; 

1 

•  FULL  ABILITY  TO  ANTICIPATE/ACCOMMODATE  TRENDS 

GOOD 

•  FULL  KNOWLEDGE  OF  A/C  ENERGY  STATE/TACTICAL 

ENVIRONMENT/MISSION; 

2 

•  PARTIAL  ABILITY  TO  ANTICIPATE/ACCOMMODATE  TRENDS; 

•  NO  TASK  SHEDDING 

ADEQUATE 

•  FULL  KNOWLEDGE  OF  A/C  ENERGY  STATE/TACTICAL 

ENVIRONMENT/MISSION; 

■x 

•  SATURATED  ABILITY  TO  ANTICIPATE/ACCOMMODATE  TRENDS; 

O 

•  SOME  SHEDDING  OF  MINOR  TASKS 

POOR 

•  FAIR  KNOWLEDGE  OF  A/C  ENERGY  STATE/TACTICAL 

ENVIRONMENT/MISSION; 

A 

•  SATURATED  ABILITY  TO  ANTICIPATE/ACCOMMODATE  TRENDS; 

•  SHEDDING  OF  ALL  MINOR  TASKS  AS  WELL  AS  MANY  NOT 

ESSENTIAL  TO  FLIGHT  SAFETY/MISSION  EFFECTIVENESS 

VERY  POOR 

•  MINIMAL  KNOWLEDGE  OF  A/C  ENERGY  STATE/TACTICAL 

ENVIRONMENT/MISSION; 

c 

•  OVERSATURATED  ABILITY  TO  ANTICIPATE/ACCOMMODATE 

TRENDS; 

•  SHEDDING  OF  ALL  TASKS  NOT  ABSOLUTELY  ESSENTIAL  TO 

FLIGHT  SAFETY/MISSION  EFFECTIVENESS 

Figure  2.  China  Lake  Situational  Awareness  (CLSA)  Scale, 


An  aggressive  flight  test/demonstration  schedule  does  not  often  permit  the  use  of  workload  metrics  like 
NASA  Task  Loading  Index  (NASA  TLX)  or  Subjective  Workload  Assessment  Technique  (SWAT)  or  situational 
awareness  metrics  like  Situational  Awareness  Rating  Technique  (SART)  Time  in  the  cockpit  (as  well  as  flight  test 
hours)  is  in  short  supply:  Workload/SA  ratings  must  be  given  quickly  and  in  a  manner  that  does  not  intrusively  affect 
what  is  already  a  potentially  Workload-intensive  and  SA-starved  environment  for  the  test  pilot.  Yet  adequate 
sensitivity,  diagnosticity,  and  construct  validity  must  be  maintained.  CLSA  is  intended  to  satisfy  these  needs. 

II.  Strengths  and  Limitations  of  CLSA 

The  CLSA  was  developed  for  use  only  in  the  circumscribed  environment  of  flight  test,  with  the  adjunct 
measures  of  the  other  resources  (Bedford  Scale,  HUD  cameras,  hot  mic,  etc.)  to  provide  de  facto  methodological 
coverage  where  the  CLSA  has  not  yet  been  formally  validated. 
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A.  Strengths 


•  High  face  validity.  When  queried,  pilots  generally  indicate  that  CLSA  jibes  with  “common  sense1'  and  the  way 
they  are  trained. 

•  Clear  criteria.  The  criteria  for  the  ordinal  scale  values  in  CLSA  are  clear  and  concise. 

•  Easily  understood.  The  intent  and  rationale  for  the  scale  are  easily  comprehended,  and  the  criteria  are  easily  used. 

•  Easily  incorporated  into  deck  of  flight  cards.  CLSA  fits  onto  one  flight  card  for  hot  mic  ratings  or  can  be  placed  on 
a  small  number  of  cards  in  case  written  ratings  are  desired, 

•  Unidimensional  ratinu  scale.  The  CLSA  format  and  protocol  are  based  on  the  Bedford  Workload  Scale. 

•  Quick  turnaround  of  data.  Data  are  gathered  quickly  and  unobtrusively,  and  can  be  gathered  pass  by  pass. 

•  Economical  and  efficient  Flight  test  dollars  and  hours  are  conserved. 

•  Easily  used  during  debrief.  HIJD  tapes  with  ratings  on  hot  mic  can  be  used  to  examine  flight  test  events  in  a 
structured  and  detailed  manner. 

•  Assumes  same  model  as  SAGAT.  SAGAT  has  been  extensively  evaluated,  though  in  the  simulator  environment, 

•  Pilots  are  Subject  Matter  Experts  (SMEs).  The  test  pilot  is  not  only  a  trained  combat  aviator  with  thousands  of 
flight  hours;  he  is  also  a  trained  test  engineer.  He  is  the  closest  thing  available  to  a  calibrated  human  observer  in 
the  cockpit.  Ele  and  only  he  gives  the  CLSA  ratings.  By  extension,  a  subjective  rating  given  by  a  pilot  becomes  as 
legitimate  a  flight  test  metric  as  any  other  employed  in  the  test  plan. 

B  Limitations 

•  Adaptation  of  simulator- based  model.  SAGAT  has  been  extensively  evaluated,  but  is  a  simulator-based  metric 
with  a  “pause  and  query”  protocol  The  CLSA  protocol  does  not  call  for  pause  since  it  is  used  during  actual  flight. 
Instead,  CLSA  relies  on  use  of  HUD  tapes  with  ratings  on  hot  mic  for  use  during  debrief.  Validation  of  this 
approach  is  required 

•  Subjective  metric.  Ideally,  pilots  should  be  briefed  on  any  subjective  metric  scale,  so  that,  as  trained  SMEs,  they 
give  ratings  based  on  a  clearly  defined  set  of  criteria  But,  CLSA  is  still  a  subjective  metric.  It  is  not  based  solely 
on  objectively  observed  events. 

•  Unidimensional  scale.  All  the  statistical  evidence  is  on  one  scale  as  opposed  to  several  subscales;  SART  and 
NASA  TLX  are  both  multidimensional 

•  Small  sample  sizes  during  flight  test.  Pilots  are  SMEs  giving  “expert”  CLSA  ratings  during  flight  test;  but  there  is 
a  limit  to  the  number  of  trained  fighter/test  pilots.  Research  is  necessarily  done  with  small  sample  sizes  and 
restricted  opportunities  of  flight  test/flight  hours. 

•  Norming.  All  subjective  rating  scales,  including  CLSA,  should  be  held  to  the  same  criteria  as  any  other 
psychometric  scales.  This  means  that  validity  and  reliability  must  be  properly  established,  and  that  the  scale  should 
be  properly  normed  against  its  intended  subject  population.  The  intended  subject  population  is  trained  test  pilots; 
the  sample  size  is  bound  to  be  small  compared  to  the  sample  sizes  usually  used  for  norming  scales. 

•  Limited  universe  of  discourse.  At  this  juncture,  CLSA  is  only  intended  for  use  in  flight  test  and  its  generalizability 
to  other  areas  is  circumscribed.  It  should  not  be  assumed  to  be  valid  when  employed  in  simulator  work  or  for  use 
in  the  real  world  until  further  studies  are  performed 
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•  Intrinsic  limitations  of  SA  metrics.  First,  any  SA  data  that  were  gathered  successfully  w'ere  obtained  from  the  pilot. 
Therefore,  he  w'as  in  a  benign  SA  environment.  Total  loss  of  SA  can  mean  loss  of  the  aircraft  or  the  pilot,  as  in  a 
Controlled  Flight  into  Terrain  (CFIT)  event.  Unless  a  safety  pilot  is  available  so  that  the  pilot  can  safely  encounter 
extremely  low  SA,  then  SA  ratings  are  limited  to  a  benign  range.  Second,  any  metric  is  used  in  an  artificial 
situation  like  flight  test  or  a  simulator;  not  real  life/field  conditions.  This  is  additionally  limited  by  mapping 
simulator  data  to  the  real  w'orld;  very  few'  actual  fatalities  have  occurred  in  a  dome. 

III.  Data  RequirementsyThresholds  for  CLSA 

Data  are  on  an  ordinal  scale  and  must  be  treated  accordingly  when  statistical  analysis  is  applied.  No  interval 

or  ratio  scale  properties  may  be  inferred.  Nonparametric  statistics  would  be  the  most  appropriate  analysis  method. 

Nonparametric  frequency  tests  such  as  Chi-Square  or  Kolmogorov-Smimov  should  be  used  when  checking  similarity 

or  difference  to  another  scale. 

IV.  Norming/Validation  Concerns  for  CLSA 

•  CLSA  has  already  been  used  with  success  in  the  Integrated  Helmet  and  Audio-Visual  System  (I  HA  VS)  flight 
demonstration;  it  w  as  developed  for  the  demonstration  to  permit  execution  of  multiple  test  events  per  flight. 

•  Comparison  of  CLSA  w'ith  accepted  SA  scales  (Tables  I  and  2,  Figure  3): 

1.  The  Situational  Awareness  Supervisory  Rating  Form  (SASRF)  was  developed  for  use  in  measuring  the 
SA  capabilities  of  F-I5  pilots.  Thirty-one  items  rate  pilot  traits  and  tactical  capabilities.  Its  format  and 
protocol  are  unsuitable  for  pass-by-pass  SA  measurement. 

2.  Crew'  Situational  Awareness  (CSA)  measures  the  SA  of  flight  crews  in  terms  of  performance  and 
coordination.  It  is  crew-based  rather  than  based  on  the  individual  pilot,  and  has  a  protocol  that  is 
unsuitable  for  pass-by-pass  SA  measurement 

3.  Situational  Aw'areness  Global  Assessment  Technique  (SAGAT)  has  a  useable  model  but  is  simulator 
based  and  has  a  protocol  unsuitable  for  pass-by-pass  SA  measurement. 

4.  Situational  Awareness  Response  Technique  (SART)  has  a  good  model,  but  its  format  and  protocol  are 
unsuitable  for  pass-by-pass  SA  measurement. 


Component 

SASRF 

CSA 

SAGAT 

CLSA 

SART 

Internal  State 

V 

External  State 

~1~ 

~T~ 

~T~ 

V 

T~ 

Relationship  of  System  to 
Environment 

V 

~T~ 

~T~ 

Environment 

~r 

V 

~T~ 

V 

Table  1.  Ability  to  measure  SA  components. 


Level 

SASRF 

CSA 

SAGAT 

CLSA 

SART 

Perceive  Environment 

V 

Comprehend  Current 
Situation 

V 

~r 

~1~ 

Project  Future  Status 

V 

~r~ 

Table  2.  Ability'  to  measure  SA  levels. 


160 


•  Situational  Awareness  has  many  competing  conceptual  and  operational  definitions,  as  does  workload.  Ultimately, 
SA  has  to  depend  on  the  operational  definition.  For  instance,  NASA  TLX  and  3D  SART  share  the  same 
dimensions.  It  can  be  inferred  that,  since  these  scales  are  intended  for  use  in  similar  situations,  that  they  have 
similar  forms  though  different  functions.  In  addition,  SASRF  has  a  different  format  and,  not  surprisingly,  also  has 
a  different  use,  function,  and  theoretical  basis.  Not  all  SA  metrics  are  alike,  and  they  are  not  interchangeable,  any 
more  than  are  all  Workload  metrics. 

•  Confounding  variables  like  motivation  or  arousal  may  cause  differences  in  subjective  scale  ratings.  Arousal  could 
affect  ratings:  overstimulation  or  understimulation  can  influence  efficiency  of  processing  and  the  degree  to  which 
the  situation  is  observed  to  be  pleasant  or  aversive  to  the  subject.  Motivation  could  affect  results  in  that  the  task 
could  be  perceived  by  the  pilot  as  irrelevant,  a  waste  of  time,  or  arduous  because  he  is  not  getting  enough  of  the 
right  information  at  the  time,  fhe  pilot  could  be  highly  motivated  and  yet  simply  be  overloaded  or  unable  to 
secure  the  right  information  in  a  timely  manner  for  the  task,  and  rate  the  situation  accordingly. 

•  Evaluation  of  CLSA  as  an  adequate  SA  model:  (Table  3). 


Table  3.  Satisfaction  of  SA  Model  Requirements  by  CLSA. 


Requirement 

Satisfaction 

Content  of  Item 

OK 

In 

Work 

TBD 

N/A 

Psychometrics 

•  Accuracy 

~T~ 

Is  output  precise? 

•  Objectivity 

~T 

Does  model  change  as  data 
changes? 

•  Reliability 

Are  there  consistent  outputs  for 
inputs? 
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•  Sensitivity 

- 1 - 

Output  calculated  at  least  5X 
input  rate? 

•  Validity 

V 

Face/concurrent/content/ 
construct/  operational  validity 

OK? 

Utility 

•  Acceptability 

V 

Acceptable  for  users? 

•  Detail 

V 

Usable  with  individuals/ 
teams/crews/separate  groups? 

•  Diagnostic 

V 

Determine  causes  of  good/bad 
SA? 

•  Flexible 

V 

Model  can  be  easily  changed 
in  future? 

•  Inexpensive 

~r 

Model  inexpensive  to  use? 

•  Provide  Quantitative 
Output 

~T~ 

Quantitative  output9 

•  Simplicity 

V 

Simple  to  administer, 
understand,  and  interpret? 

Human  Characteristics 

•  Attention 

V 

Any  limitations  to  attention  from 
any  source? 

•  Perception 

V 

Any  limitations  for 
characteristics  of  perception? 

•  Memory 

V 

Limitations  to  working 
memory? 

•  Automaticity 

V 

limitations  to  automaticity/ 
accommodation  of  change? 

•  Goal-Driven  or  Data- 
Driven  Processing 

V 

Does  model  discriminate 
between  goal-dnven  and  data- 
dnven  processing? 

•  Individual  Differences 

V 

Does  individual  processing/ 
coping  make  a  difference? 

System  Characteristics 

•  System  Design 

V 

Does  the  system  provide 
everything  operator  needs 
when  needed? 

•  Stress  and  Workload 

V 

Are  stress  and  workload 
limitations  to  model? 

•  Complexity 

V 

Is  model  complexity  a  limiting 
factor? 

•  Automation 

V 

Is  degree  of  system 
automation  a  limiting  factor? 

Table  3.  Satisfaction  of  SA  Model  Requirements  by  CLSA.  (Continued) 


V.  Future  Efforts 

During  CLSA  development,  this  researcher  was  forced  to  delay  some  of  the  methodological  steps  needed  to 
validate  CLSA  as  a  subjective  rating  scale  in  order  to  fit  it  into  the  1HAVS  flight  test  regime.  The  functions  performed 
by  these  methodological  steps  were  taken  up  by  reliance  on  structured  debriefing  techniques/expanded  protocols:  in 
particular,  structured  interviews,  HMD  videotapes,  hot  mic,  and  detailed  questionnaires  were  used  to  address  the 
functions  of  Accuracy,  Objectivity,  Diagnosticity,  Goal/Data  Driven  Processing,  Individual  Differences,  and 
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Automation.  Therefore,  the  delayed  Concurrent  (e.g.,  alternative  measure)  and  Construct  (e.g.,  internal  state  measures) 
Validities  shall  be  addressed  by  proposed  experiments. 

These  proposed  experiments  will  employ  an  existing  SA  measure  (3D-SART)  and  a  neuropsychological  test 
(NASA  Spaceflight  Cognitive  Assessment  Tool  (S-CAT))  concurrently  w ith  CLSA  in  statistical  validation  studies. 
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INTRODUCTION 

To  date,  virtual  environments  applied  to  aviation  training  have  been  created  under  the  assumption  that 
flying  is  flying  and  that  tools  to  train  pilots  are  basically  independent  of  the  type  of  aircraft  in  question  This  is  in 
fact  not  the  case  at  all.  Most  of  the  training  tools  that  have  been  developed  over  the  years  for  the  fixed  wing 
community  are  fundamentally  inappropriate  for  the  rotary  wing  community.  Systems  such  as  TOPSCENE  [11, 
which  has  been  widely  regarded  as  an  excellent  mission  rehearsal  system  for  fixed  wing  pilots,  is  seldom  even 
used  by  rotary  wing  pilots  because  it  does  not  represent  the  information  helicopter  pilots  need  in  the  form  that 
they  will  see  it.  High  and  fast  is  different  from  low  and  slow.  These  differences  must  be  accounted  for  in  training 
systems  for  helicopter  pilots  by  providing  tools  specifically  designed  for  their  needs  and  missions. 

Navigation  is  an  essential  component  of  all  helicopter  missions.  While  most  training  time  is  spent  on 
mission-specific  items  and  procedures,  all  goes  for  naught  if  the  pilot  cannot  find  the  target.  Surprisingly, 
navigation  does  not  receive  much  attention  at  all  in  the  training  curriculum.  At  Helicopter  Antisubmarine 
Squadron  Ten  (HS-10),  the  SH-60F  and  HH-60H  f  leet  Replacement  Squadron  (FRS),  three  days  of  the  160  day 
curriculum  are  spent  on  Combat  Search  and  Rescue  (CSAR)  ground  school,  a  part  of  which  is  MITAC  (Map 
Interpretation  and  Terrain  Association  Course).  Of  course,  with  such  little  emphasis  spent  on  map  reading  and 
navigation,  pilots  tend  to  struggle  with  the  two  flights  they  must  make  to  demonstrate  mastery  of  these  skills.  A 
lot  of  time  is  spent  in  the  air  circling  checkpoints,  recovering  from  errors,  and  generally  showing  student  pilots 
that  flying  over  Southern  California  is  very  different  from  flying  over  Pensacola,  Florida,  where  they  conduct 
initial  terrain  navigation  training. 

Background 

Historically,  CSAR  ground  school  was  a  two  week  long  school  and  MITAC  training  received  much 
greater  emphasis.  However,  even  then,  training  techniques  were  less  than  optimal,  and  in  some  cases, 
counterproductive.  Many  of  the  pilots  in  the  fleet  today  received  MITAC  training  from  a  videotape  filmed  from 
a  Huey  UH-1  and  played  back  at  about  double  speed  The  task  was  to  watch  the  video  and  chart  progress  with  a 
grease  pencil  on  a  laminated  contour  map.  This  type  of  training  has  a  number  of  shortcomings.  With  the  limited 
field  of  view  available  on  a  videotape,  pilots  do  not  learn  how  to  use  peripheral  vision  to  maintain  orientation. 
At  accelerated  speeds,  pilots  learn  that  the  only  way  to  maintain  position  on  the  map  is  to  pick  out  landmarks 
(peaks  or  other  terrain  features)  at  great  distances.  This  causes  them  to  stop  paying  attention  to  what  is  right  in 
front  of  the  aircraft.  There  is  also  a  complete  lack  of  interaction  in  the  video.  If  a  mistake  is  made,  there  is  no 
way  for  the  student  to  go  back  and  determine  what  caused  it  This  is  a  fundamental  part  of  learning 

What  has  resulted  is  a  situation  where  navigation  skills  today  are  almost  entirely  learned  in  the  aircraft. 
Considering  the  maintenance  costs  of  a  typical  military  helicopter,  this  is  an  expensive  way  to  learn.  Even  if  it 
wasn't  so  expensive,  there  is  another  problem.  Flight  instructors  at  HS-10  are  restricted  as  to  what  routes  they 
can  fly  They  are  not  free  to  make  up  new  routes  at  will.  Consequently,  once  a  student  pilot  has  flown  a  route,  all 
map  and  terrain  interpretation  stops  It  then  becomes  a  memory  or  landmark  recognition  task  of  "How  did  1  do 
this  last  time?"  rather  than  map  reading.  The  ability  to  fly  unique  routes  would  greatly  enhance  the  flight 
instructor's  ability  to  teach  this  skill.  Even  if  alternate  routes  were  available,  flight  instructors  would  not  be  able 
to  evaluate  student's  navigation  ability  over  terrain  types  other  than  Southern  California  and  Arizona.  How  well 
can  they  navigate  over  the  desert?  How  well  can  they  navigate  over  relatively  featureless  terrain?  Navigation 
training  in  the  air  is  not  only  excessively  expensive,  but  also  limited  in  effectiveness. 

Helicopters  are  flown  by  two  pilots  —  one  maintains  control  of  the  aircraft  and  is  responsible  for  avoiding 
hazards  (e.g.  power  lines  and  vegetation)  and  for  verbally  identifying  features  for  the  navigator  who  is 
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responsible  for  charting  the  current  position  and  for  guiding  the  flying  pilot  This  is  typically  done  with  verbal 
commands.  The  important  factor  here  is  that  the  navigating  pilot  is  not  doing  the  flying.  These  pilots  already 
know  how  to  fly.  Our  objective  is  not  to  teach  flying  but  to  teach  navigation  skills.  Therefore,  if  a  single  pilot  is 
to  learn  how  to  navigate,  th^  interface  to  the  training  system  must  have  no  learn  mg  curve  associated  with  it.  It 
should  be  as  near  to  "walk  up  and  use"  as  possible. 

Requirements 

In  summary,  what  HS-IO  needs  is  a  way  to  allow  student  pilots  to  fly  unique  routes  over  real  (or 
topologically  similar)  terrain  while  reading  a  contour  map.  They  must  be  able  to  review  their  flights  to  get 
appropriate  feedback  as  to  where  mistakes  were  made.  They  should  not  be  left  in  a  disoriented  condition  for 
prolonged  periods  of  time.  This  causes  frustration,  diminished  self  confidence,  and  is  otherwise  not  helpful  to 
the  training  process.  Ideally,  much  of  this  can  and  should  be  done  outside  of  instructor  view.  Students  who  are 
not  adept  at  navigation  skills  know  it.  If  they  had  a  way  to  develop  and  hone  their  skills  before  a  graded  flight, 
they  would  certainly  do  so.  However,  no  such  mechanism  currently  exists  other  than  extra  map  study.  The 
interface  to  this  system  must  be  simple  to  use.  It  doesn’t  need  to  be  like  real  flying  since  the  navigator  doesn't  do 
the  flying  anyway.  This  training  capability  must  be  made  available  at  the  squadron  level.  If  an  expensive  large- 
scale  solution  were  to  be  developed,  it  would  not  be  used  due  to  a  lack  of  availability  by  individual  pilots  on  an 
as-needed  basis. 

APPROACH 

There  are  a  number  of  practical  constraints,  in  addition  to  those  defined  by  the  needs  of  HS-IO,  to 
constructing,  evaluating,  and  eventually  fielding  a  trainer  of  this  type.  Since  the  system  must  be  available  on  an 
as-needed  basis  at  the  squadron  level,  it  must  therefore  be  relatively  inexpensive  and  easy  to  maintain.  HS-IO 
does  not  have  the  manpower  nor  the  financial  resources  to  accommodate  another  large  expensive  training 
system  in  addition  to  the  full  motion  flight  simulators  they  already  maintain.  We  envisioned  a  small  system  that 
could  occupy  a  comer  in  a  classroom  or  ready  room.  This  would  allow  for  asynchronous  training  to  occur  -  the 
student  can  use  the  system  without  instructor  intervention. 

Ideally,  the  system  would  be  implemented  on  general  purpose  hardware  This  would  make  it  easier  to 
maintain  and  develop  further.  The  current  implementation  almost  achieves  this  goal.  It  is  small  and  transportable 
but  uses  specialized  graphics  hardware.  We  felt  that  this  was  a  reasonable  compromise  at  this  time  since  PC 
graphics  hardware  is  improving  at  such  a  rapid  pace.  We  believe  that  in  the  time  it  takes  us  to  determine  what 
the  system  has  to  look  like  and  we  evaluate  that  it  is  effective,  the  time  will  be  right  to  port  the  system  to  a 
graphics  PC.  This  work  is  in  its  early  stages  at  this  time. 

After  determining  what  the  general  training  need  was  at  HS-10,  wc  developed  a  rough  implementation 
and  brought  it  to  HS-IO  for  their  feedback  We  learned  that  they  specifically  did  not  want  a  mission  rehearsal 
system  like  TOPSCENE  but  rather  needed  something  to  help  students  learn  to  read  contour  maps.  As  we  worked 
on  the  second  iteration,  we  used  students  at  the  Naval  Postgraduate  School  and  specifically  the  Aviation  Safety 
School  in  several  usability  tests  to  work  out  the  details  of  the  interface.  At  this  point,  the  system  was  brought  to 
HS-IO  to  begin  data  collection  on  the  actual  training  effectiveness  experiment. 

IMPLEMENTATION 

The  prototype  navigation  trainer  was  implemented  on  an  lndigo2™  graphics  workstation  from  Silicon 
Graphics,  Inc.  (SGI).  The  system  contains  a  single  R4400  200  MHz  CPU  with  128  Mbytes  of  RAM,  a  High 
Impact™  graphics  board,  and  IMPACT™  Channel  Option  Board  to  allow  the  use  of  multiple  graphics  monitors 
from  a  single  machine.  The  display  setup  uses  three  19"  monitors  in  a  semicircular  configuration.  The  three 
monitors  provide  about  95°  field  of  view  when  sitting  27"  from  the  screens.  The  control  device  is  a  Flybox™ 
from  BG  Systems,  Inc.  Figure  1  shows  the  basic  configuration  of  the  system.  The  fourth  monitor  shown  is  used 
as  a  console  and  is  not  used  as  a  display  during  training. 
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Figure  1 .  The  apparatus  for  the  prototype  helicopter  navigation  trainer. 


Software 

The  software  was  developed  entirely  at  the  Naval  Postgraduate  School  using  the  Performer™  application 
programmers  interface  (API)  from  SGI.  At  its  heart  is  a  simple  terrain  fly  through  program  augmented  with  the 
necessary  maps  and  gauges  required  for  this  application.  The  flight  control  was  designed  to  be  as  simple  as 
possible.  We  use  a  terrain  following  technique  such  that  the  pilot  sets  a  course  (bearing),  altitude  above  ground 
level  (AGL),  and  ground  speed  in  knots.  There  is  a  minimum  altitude  of  50'  AGL  so  that  crashing  into  the 
ground  is  impossible.  There  is  no  maximum  altitude.  The  pilot  can  then  look  at  the  map  or  attend  elsewhere  as 
the  virtual  helicopter  flies  itself.  There  are  absolutely  no  aerodynamics  applied  to  the  flight  model.  This  is  not 
what  we  are  training. 

The  display  is  divided  into  three  VGA  resolution  screens,  one  for  each  monitor.  However,  there  is  a  7° 
gap  between  each  of  the  monitors  for  the  plastic  casing.  We  account  for  this  by  leaving  a  7°  gap  in  the  graphics 
rendering.  Consequently,  as  a  pixel  leaves  one  monitor,  it  does  not  instantly  jump  the  gap  to  the  next.  These 
gaps  are  analogous  to  the  aircraft’s  vertical  windscreen  support  frames.  Figure  2  shows  a  typical  three  screen 
view  with  gaps  between  screens. 


Figure  2.  A  typical  three  screen  view  with  gaps  between  monitors. 


Interface 

The  interface  was  designed  to  be  easy  to  learn  and  consistent  with  the  task  of  moving  a  viewpoint  through 
the  terrain  as  practice  for  contour  map  interpretation  and  visualization.  However,  there  is  one  basic  problem: 
Flying  is  generally  a  two-handed  task  Map  reading  and  interpretation  is  at  a  minimum  a  one-handed  task  under 
the  best  of  conditions.  In  the  aircraft,  a  pilot  would  never  attempt  both  tasks  simultaneously.  For  training,  our 
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contention  is  that  interactive  control  is  important  [2].  We  did  not  want  pilots  to  feel  like  passive  passengers  but 
rather  active  participants.  Therefore  the  interface  needed  to  support  completing  both  tasks  simultaneously.  It  was 
important  to  design  an  interface  that  could  be  operated  as  easily  by  a  single  pilot  as  by  a  pair  of  pilots. 

Although  we  had  a  wealth  of  specific  domain  knowledge  actively  involved  in  this  project,  we  decided 
to  bring  the  problem  and  our  ideas  to  HS-IO  and  the  helicopter  pilot  community  at  large  to  find  out  if  we  were 
on  the  mark.  When  helicopter  pilots  were  presented  with  stick  and  throttle  type  controls,  they  were  nearly  split 
on  which  device  should  control  each  axis  of  movement  The  transformation  of  stick  and  throttle  to  cyclic  and 
collective  is  ambiguous.  A  narrow  majority  believed  the  throttle  should  act  as  a  collective.  When  the  stick  is 
pulled  back  toward  the  user,  the  model  should  respond  as  a  helicopter  would  if  the  collective  were  raised.  A 
slight  minority  believed  the  throttle  should  act  as  a  fixed  wing  throttle,  i.e.  it  should  control  forward  speed 
Clearly,  the  results  of  our  flight  control  usability  study  suggested  our  goal  of  building  a  system  that  was  easy  to 
use  for  everyone  was  not  entirely  achievable.  After  looking  at  our  target  user  group,  we  still  did  not  have  a 
definitive  answer  to  the  question  of  control  mechanism.  We  decided  to  look  not  only  at  the  user  group  at  large 
but  also  at  the  user  group  executing  the  training  task  at  HS-10.  This  study  brought  out  an  issue  we  had  not 
considered.  If  students  rely  on  dead  reckoning  (DR)  techniques,  maintaining  a  constant  ground  speed  should 
require  little  or  no  cognitive  workload.  How  ever,  if  the  speed  is  set  by  the  cyclic,  maintaining  a  constant  ground 
speed  requires  excessive  cognitive  workload  The  final  compromise  was  to  adopt  a  fixed  wing  mode  where 
cyclic  controls  climb  and  yaw  while  throttle  controls  speed  There  are  no  flight  dynamic  characteristics 
associated  with  the  helicopter  model  motion.  However,  it  was  decided  that  extraordinary  motion  seemed  like  a 
good  way  to  compensate  for  inherent  limitations  of  training  media.  Since  we  can’t  provide  the  same  horizontal 
and  vertical  field  of  view  of  the  real  aircraft  and  are  restricted  with  limited  model  fidelity,  we  can  attempt  to 
make  up  for  such  shortcomings  by  allowing  users  to  do  things  only  possible  in  a  virtual  world  --  specifically 
flying  backwards  and  the  ability  to  detach  the  viewpoint  from  the  helicopter. 

The  F  xocentric  View 

While  navigating,  we  typically  only  have  an  egocentric  view  available  to  us.  This  is  our  individual  view 
from  where  we  currently  are  looking  through  our  own  eyes.  Previous  research  suggests  that  an  exocentric  view 
is  a  useful  mechanism  for  acquiring  information  about  a  large-scale  space  [3,  4]  An  exocentric  view  is  one 
where  the  view  is  detached  from  the  position  of  the  egocentric  view  but  is  not  necessarily  perspective-less 
(infinitely  far  away  and  directly  above)  as  would  be  the  case  in  a  conventional  map.  This  view  can  be  useful  for 
navigation  because  it  shows  the  local  context  around  the  viewpoint  without  losing  perspective. 

We  originally  considered  using  a  "wingman"  camera  position  tethered  to  the  virtual  helicopter.  This 
was  discarded  because  it  fundamentally  changed  the  navigation  task.  It  is  important  that  movement  take  place  in 
the  egocentric  view  only.  The  exocentric  view  was  meant  to  provide  help  when  needed  but  we  feared  it  would 
become  a  crutch  which,  when  taken  away  for  the  actual  flight,  would  actually  serve  to  lower  performance  rather 
than  raise  it.  We  also  considered  a  separate  window  for  the  wingman  view  but  also  discarded  it  for  similar 
reasons.  If  an  exocentric  view  was  to  be  used,  it  was  essential  that  the  transformation  from  it  to  the  egocentric 
view  be  completely  obvious. 

We  finally  decided  to  integrate  the  exocentric  and  egocentric  views.  To  minimize  problems  of 
disorientation  associated  with  teleportation  (e.g.,  a  discontinuous  transition),  we  decided  on  a  fluid  transition 
from  Ihe  egocentric  to  the  exocentric  perspective.  It  is  necessary  that  the  user  remain  oriented  throughout  a 
training  session.  We  developed  a  metaphor  whereby  the  user  detaches  the  camera  from  the  model  and  controls 
the  viewpoint  with  the  flight  stick.  Holding  the  cyclic  trigger  switch  while  pulling  back  on  the  stick  detaches  the 
camera  from  the  helicopter  and  moves  it  up  a  shallow  10°  slope  away  from  the  helicopter.  The  viewpoint's  speed 
of  movement  away  from  the  helicopter  is  proportional  to  the  stick  displacement  and  distance  squared  from  the 
helicopter.  The  viewpoint  can  be  rotated  about  the  helicopter  by  pushing  the  flight  stick  either  left  or  right. 
When  the  trigger  switch  is  released,  the  viewpoint  reverses  the  path  the  user  controlled  and  returns  to  the 
egocentric  view.  The  animated  motion  is  fluid  and  continuous.  Figures  3  and  4  show  exocentric  views  at  two 
points  along  the  glide  slope.  In  Figure  3,  the  viewpoint  has  just  been  detached.  The  virtual  helicopter  model  has 
been  highlighted  in  this  image.  In  Figure  4,  the  user  has  pulled  further  away  such  that  the  helicopter  model  has 
been  replaced  by  its  symbol.  This  has  also  been  highlighted 
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Figure  3.  An  exocentric  view  shortly  after  the  view  Figure  4.  An  exocentric  view  after  the  user  has 


was  detached  backed  away  from  the  helicopter  position. 

The  Heads-Up-Disnlay 

The  Heads-Up-Display  (HUD)  used  is  extremely  simple.  Again,  we  did  not  make  any  attempt  to 
replicate  the  actual  cockpit  displays.  We  determined  the  essential  flight  parameters  of  interest  to  the  terrain 
navigator  and  provided  only  that  information:  Mean  Sea  Level  (MSL)  altitude  in  feet,  Above  Ground  Level 
(AGL)  altitude  in  feet,  true  heading  in  degrees,  and  ground  speed  in  knots.  Figure  5  shows  an  enlarged  image  of 
the  HUD  from  Figure  4.  The  display  is  red  over  mostly  brown  and  green  terrain  and  is  therefore  easier  to  read 
than  might  be  suggested  here  in  grayscale.  In  this  example,  the  heading  is  78°,  the  ground  speed  is  0  knots,  and 
the  altitude  is  1 73  U  above  sea  level  or  285’  above  the  ground  at  this  point. 


Figure  5.  The  Head-Up-Display. 


The  You-Are-Here  Map 

It  is  essential  that  the  student  pilot  not  be  lost  for  any  extended  duration  during  a  training  session  If  the 
pilot  becomes  disoriented,  pressing  the  spacebar  calls  up  a  You  are  Here  (YAH)  map.  This  map  is  a  digital 
replica  of  the  paper  map  they  are  using.  The  YAH  map  window  can  be  moved,  resized,  and  iconified  The 
spacebar  toggles  the  YAH  map  on  and  off.  When  the  YAH  map  is  displayed,  helicopter  motion  is  frozen  If  the 
student  could  view  the  map  and  move  concurrently,  the  map  would  become  a  crutch.  We  observed  pilots  in 
earlier  evaluations  flying  exclusively  off  of  the  map,  not  attending  to  the  primary  displays  at  all  By  halting 
motion  while  the  map  is  displayed,  we  have  eliminated  this  counterproductive  strategy.  In  principle,  the  map 
acts  like  standard  moving  map  display.  The  helicopter  remains  centered  with  the  map  oriented  in  the  helicopter’s 
direction  of  travel  (track-up).  This  is  consistent  with  previous  work  showing  that  track-up  maps  are  most 
appropriate  for  egocentric  tasks  such  as  active  navigation  [5,  6].  This  is  reinforced  by  instructors  at  HS-10  who 
direct  students  to  always  turn  their  maps  in  the  direction  of  travel.  The  symbology  on  the  map  includes  the 

intended  track  for  various  training  routes  (only  one  at  a  time),  the  own  ship  path  showing  where  the  virtual 
helicopter  has  been,  and  an  icon  representing  the  current  position  and  orientation.  The  user  can  control  the  zoom 
factor  on  the  map  with  a  lever  on  the  Flybox™.  Pushing  the  lever  forward,  away  from  the  user,  zooms  in  while 
pulling  back  on  the  lever  zooms  out.  Other  map-related  functions  available  to  the  user  include  going  back  to  the 
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last  point  at  which  they  checked  the  map,  erasing  their  track,  selecting  a  different  training  route,  and  returning  to 
the  starting  point.  Figure  6  shows  the  YAH  map  with  track  data  and  helicopter  symbol.  The  intended  track  is 
black  while  the  actual  track  and  helicopter  symbol  are  red. 


Figure  6.  The  You-Are-Here  map. 


The  Terrain  Database 

The  area  we  chose  to  model  for  the  prototype  system  is  the  same  terrain  that  student  pilots  will  actually 
fly  over  —  specifically  Marine  Corps  Air  Station  (MCAS)  Camp  Pendleton,  California.  However,  we  chose 
routes  through  this  environment  that  differ  significantly  from  those  used  by  flight  instructors.  The  modeled  area 
is  an  1 8  by  21  nautical  mile  (NM)  region  bounded  by  N33.25  W1 17.60  and  N33.35  W1 1 7.25.  We  first  obtained 
the  Digital  Elevation  Terrain  Data  (DTED)  for  this  area  from  the  National  Imagery  and  Mapping  Agency 
(N1MA).  We  used  DTED  level  one  which  is  100  meter  resolution,  unclassified,  and  publicly  available  This  was 
imported  into  the  EasyTerrain™  terrain  modeling  tool  by  Coryphaeus,  Inc,  to  produce  a  polygonal  model.  We 
then  obtained  geo-rectified  multi-spectral  satellite  imagery  from  the  Naval  Space  Command  which  we  used  for 
texture  over  the  polygonal  model.  The  resolution  of  the  satellite  imagery  is  30  meters  and  is  also  unclassified 
and  publicly  available.  The  imagery  texture  works  well  for  distant  views  or  high  fly-over  applications  such  as 
those  typical  of  TOPSCENE.  However,  from  low  altitudes,  the  local  area  appears  very  pixelated  with  large 
colored  blocks  where  texture  pixels  are  spread  over  a  large  region.  This  lack  of  realism  did  not  disturb  us  until  it 
was  determined  that  it  had  a  strong  negative  effect  on  navigation  performance.  It  became  difficult  to  determine 
relative  ground  speed  via  optical  flow.  This  makes  dead  reckoning  techniques  difficult  if  not  impossible.  To 
remedy  this,  we  added  a  detailed  texture  which,  when  viewed  up  close,  overlays  the  imagery  texture.  The  detail 
texture  was  created  using  the  random  noise  generation  features  of  Adobe  Photoshop™.  Colors  were  selected  to 
match  the  general  appearance  of  Camp  Pendleton  foliage  —  dry  grass  and  low  chaparral.  This  texture 
management  technique  is  intended  to  preserve  the  quality  of  information  available  from  satellite  imagery  for 
distant  terrain  while  improving  the  appearance  of  terrain  close  to  the  viewer.  We  do  not  believe  that  detailed 
texture  completely  eliminates  problems  associated  with  ground  speed  estimation.  However,  it  was  made  clear 
through  our  usability  tests  that  the  task  was  not  always  possible  without  the  use  of  detailed  texture. 

We  did  not  include  vegetation  in  the  model.  Our  intent  was  to  focus  on  terrain  navigation.  Vegetation 
can  be  used  as  a  landmark.  After  a  landmark  can  be  recognized,  there  is  no  longer  a  reason  to  resolve  the 
contour  map  to  cockpit  views.  In  other  words,  map  reading  stops  and  the  navigation  skills  necessary  to  be  a 
successful  pilot  do  not  develop  further.  If  we  add  vegetation  in  the  future,  it  will  be  to  add  velocity,  height,  and 
depth  of  field  cues.  We  will  not  try  to  replicate  the  real  world  We  believe  that  randomly  placed  vegetation  of 


170 


uniform  size  and  appearance  may  be  a  great  benefit  in  helping  pilots  use  dead  reckoning  techniques  which  are  an 
essential  part  of  resolving  terrain  to  contour  maps. 

For  similar  reasons,  very  few  man-made  features  are  modeled.  While  military  pilots  are  trained  not  to 
use  cultural  features  such  as  roads  and  power  lines  for  navigation,  they  do.  It  is  not  our  intent  to  reinforce  bad 
habits.  However,  during  our  usability  tests,  we  discovered  that  in  practice,  power  lines  are  far  too  important  a 
hazard  to  not  be  modeled.  Therefore  we  made  exceptions  to  add  primary  roads  and  power  lines  to  the  model. 

EXPERIMENT 

To  determine  if  our  prototype  implementation  was  actually  improving  navigation  performance,  we 
designed  an  experiment  with  the  cooperation  of  HS-10  to  evaluate  student  pilots’  navigation  abilities.  Although 
there  are  probably  other  ways  of  determining  if  transfer  of  training  is  occurring,  we  decided  that  the  only 
reasonable  measure  was  to  use  conventional  evaluation  techniques  already  in  place.  However,  the  curriculum  at 
HS-10  is  extremely  full  so  we  had  to  fit  our  experiment  into  their  schedule  as  time  would  allow.  The  positive 
side  of  this  is  that  we  did  not  disturb  the  training  process,  so  if  an  improvement  is  detected,  we  can  be  confident 
that  it  was  due  to  differences  we  introduced  (e.g.,  the  VE  trainer)  The  negative  side  of  this  is  that  we  had  less 
control  than  we  would  have  liked  In  particular,  we  were  not  allowed  in  the  aircraft  during  the  training  flight. 
Consequently,  there  may  be  variability  in  instructor  evaluation  we  cannot  attribute  to  student  performance. 

There  were  two  experimental  groups  of  twelve  students  each;  the  control  group  which  received  only 
conventional  map  study  and  ground  school  preparation,  and  the  virtual  environment  group  which  received  the 
same  preparation  as  the  control  group  in  addition  to  one  hour  on  the  virtual  environment  trainer.  As  of  the 
writing  of  this  paper,  six  CAT-1  pilots  have  completed  training  in  the  virtual  environment  group.  CAT-1  pilots 
are  recent  flight  school  graduates  who  are  not  switching  platforms  but  are  entering  their  first  graduate  level 
instruction  specializing  in  one  helicopter  -  in  this  case,  the  H-60  Seahawk.  Our  students  are  all  males,  ages  22  to 
25,  with  ranks  of  LTJG  (0-2)  to  LT  (0-3). 

After  an  introductory  brief  of  the  procedure  and  instructions,  participants  in  the  virtual  environment 
group  were  given  a  quick  test  to  ensure  that  they  understood  and  were  able  to  use  the  system  properly  and  that 
they  were  familiar  with  the  available  features.  Only  one  planned  route  was  given  in  this  experiment.  This  route 
is  depicted  on  both  the  virtual  and  paper  maps.  They  were  instructed  to  try  to  fly  the  route  as  closely  as  possible 
We  measured  the  number  of  times  the  virtual  map  was  accessed  and  the  location  of  the  virtual  helicopter.  A 
“perfect”  run  is  one  in  which  the  student  flies  the  route  without  mistakes  and  does  not  need  to  access  the  virtual 
map  or  exocentric  view  at  any  time  -  all  navigation  is  done  off  of  the  paper  map  alone.  This  is  the  best  case 
since  this  is  what  will  occur  later  in  the  aircraft.  However,  students  are  not  discouraged  from  accessing  the 
virtual  map.  A  “think  aloud”  verbal  protocol  was  used  to  gather  qualitative  information  about  confidence  and 
strategies  used  by  the  student.  This  is  far  more  indicative  of  navigation  abilities  than  quantitative  measures 
because  a  disoriented  or  confused  student  cannot  describe  upcoming  terrain  features  with  any  accuracy.  It  is 
possible,  however,  to  guess  correctly  as  to  which  way  to  go 

The  control  group  receives  only  conventional  map  preparation.  They  are  given  1:50,000  topological 
maps  of  the  Camp  Pendleton  area.  They  are  told  the  route  they  will  fly.  There  are  a  number  of  checkpoints  they 
must  pass  along  the  way  Their  task  is  to  familiarize  themselves  with  the  area  via  the  map  such  that  they  will  be 
able  to  navigate  in  the  aircraft.  Conventional  preparation  does  not  include  any  three-dimensional  tools 
whatsoever. 

All  participating  pilots  are  evaluated  in  an  identical  fashion.  When  in  the  aircraft,  they  are  the 
navigating  pilot.  It  is  their  task  to  direct  the  flight  instructor  who  is  the  flying  pilot.  The  flight  instructor  will 
usually  ask  questions  about  features  they  see  to  determine  if  the  student  is  cognizant  of  the  surrounding  area. 
Students  are  asked  to  describe  upcoming  features  and  cues  used  to  direct  flight.  Following  the  flight,  the  flight 
instructor  will  evaluate  the  student  as  usual  on  a  typical  grade  card.  These  cards  ask  specific  questions  about 
performance  on  the  flight  -  in  this  case  on  navigation  ability.  In  addition,  we  added  a  number  of  questions  on  a 
grade  card  addendum  that  are  specific  to  this  experiment  (Figure  7). 
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TERF  EVALUATION  RESULTS 


TERF  EVALUATION  CRITERIA 


Overall  Terrain  Navigation  Performance 

BA  4A 


Number  of  Errors 

(Mis identified  features,  check  points,  wrong  turns) 


Error  Recovery 

BA  AA 


Terrain  Feature  Identification 

BA  AA 


Value  ofTerf  Nav  Training  Timc/FRP  Progress 

BA  AA 


Overall  Terrain  Navigation  Pefformanc^r 
BA  Relied  heavily  or  entirely  on  DR  lechniques. 

Spent  a  Mgreficanl  amount  of  time  lo si 
Had  significant  difficulty  maintaining  orientation 
AA  Correctly  idoHified  terrain  features  necessary  to  nraintam  track 
Arrived  al  checkpoints  within  30  secs. 

N 'amber  of  Errors 

Wrong  luma  Mmdantified  critical  feature*  Required  correction  by  IP 
Error  Recovery 

BA  Required  significant  time  and  guidance  to  regain  orientation 
AA'  Regainod  orientation  with  minimal  cues 

Terrain  Feature  Identification 

BA  Required  significant  timohelp  to  identify  critical  features. 

AA.  Consistently  relied  on  terrain  features  to  maintain  orientation 

Value  of  Time/Student  progress 

BA  Spent  significant  amount  of  time  on  fundamental  skills 

FRP  overwhelmed  with  navigation,  little  time  for  other  tasks 
Marginally  improved  navigation  skill « 

AA-  Showed  significant  improvement  in  navigation  skills 

Comments: 

Any  additional  comments  relating  to  student’s  performance 


Comments 


Figure  7.  The  grade  card  addendum  used  for  the  experiment. 


RESULTS 

Without  a  control  group  to  compare  to,  it  is  premature  to  draw  strong  conclusions  about  student  performance 
in  the  aircraft.  However,  based  on  students’  interaction  with  the  training  system,  we  can  draw  several  significant 
conclusions. 

It  is  feasible  to  use  an  unaltered  pre-existing  task  (training  CSAR  skills  to  helicopter  pilots)  as  the  measure  of 
effectiveness  of  a  proposed  training  aid  At  the  outset,  it  was  uncertain  if  it  would  be  possible  to  implement  and  measure 
the  effectiveness  of  a  terrain  navigation  training  aid  without  altering  the  existing  training  syllabus.  While  coordinating 
the  implementation  around  a  fixed  training  course  presents  many  unique  logistic  challenges,  it  eliminates  all  questions 
related  to  training  transfer.  If  navigation  performance  in  the  aircraft  improves  for  those  students  who  use  the  navigation 
trainer,  there  can  be  no  question  concerning  its  effectiveness.  Although  there  is  considerably  more  overhead  associated 
with  data  collection  and  less  empirical  data  to  study,  improved  student  performance  in  the  aircraft  is  the  ultimate  goal  of 
a  training  system,  and  was  precisely  what  we  measured.  We  believe  this  is  the  ideal  situation  since  it  raises  few 
questions  regarding  effectiveness  and  definitively  answers  the  one  question  we  needed  an  answer  to.  A  downside  to  this 
method  is  that  it  does  not  lend  itself  easily  to  discovering  unanticipated  effects  that  might  lead  to  even  better  training 
systems  in  the  future.  We  need  to  rely  on  flight  instructors  as  the  final  judge  and  jury  of  the  system.  We  were  not 
present  in  the  aircraft  ourselves. 

The  task  of  navigating  through  a  model  of  Camp  Pendleton  is  an  achievable  goal.  This  was  not  a  foregone 
conclusion  prior  to  evaluating  students  at  HS-10.  Although  several  helicopter  pilots  familiar  with  the  Camp  Pendleton 
area  felt  comfortable  navigating  through  the  virtual  model,  they  relied  primarily  on  memory  rather  than  the  feedback 
provided  by  the  system.  While  it  was  encouraging  that  pilots  were  able  to  accurately  identify  their  location  based 
exclusively  on  the  forward  field  of  view  provided  in  the  simulation,  this  did  not  tell  us  what  would  happen  when  people 
without  prior  exposure  to  the  Camp  Pendleton  area  were  tasked  with  navigating  through  the  model.  One  of  the  key 
areas  of  uncertainty  is  whether  the  digitally  recreated  contour  map  correlated  closely  enough  with  the  scene.  All  six 
students  tested  were  able  to  complete  the  depicted  route  within  the  allotted  time.  Additionally,  after  initially  completing 
the  route,  four  of  the  six  students  were  able  to  either  complete  the  route  in  the  reverse  direction  or  repeat  the  route  in  the 
same  direction  a  second  time.  Furthermore,  it  was  clear  from  verbal  data  that  they  were  working  to  resolve  the  map  to 
what  they  saw  on  screen.  They  were  not  simply  trying  to  get  familiar  with  the  region  in  question.  Students  commented 
on  the  shape  of  the  ground  around  them  in  great  detail.  They  stated  well  ahead  of  time  what  they  expected  to  see  and 
where  it  would  appear.  When  they  made  errors  or  drifted  off  course,  they  were  able  to  quickly  recover  by  resolving 
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what  they  saw  on  screen  to  the  map  rather  than  vice  versa.  This  is  taught  in  flight  school  as  working  “outside-in”  rather 
than  “inside-out”.  The  navigator's  eyes  should  be  outside  as  much  as  possible,  not  inside  scanning  the  map 

This  study  also  validated  the  fact  that  the  interface  and  feedback  are  effective.  Although  the  interface  had  been 
evaluated  d”nng  usability  studies,  it  was  never  evaluated  with  the  ^recise  target  user  group.  V  n  usability  study  used 
non-aviators  (including  USMC  Infantry,  USN  Surface  Warfare  and  Supply  Corps  Officers),  fixed  wing  aviators  and 
helicopter  pilots.  Although  it  may  be  assumed  that  the  helicopter  pilots  would  provide  the  closest  approximation  to  the 
target  user  group,  there  were  significant  differences  between  the  pilots  tested  and  the  ultimate  user  group.  The 
minimum  flight  time  of  the  helicopter  pilots  tested  was  approximately  1500  hours.  All  had  extensive  fleet  experience 
with  overland  terrain  navigation  One  of  these  subjects  successfully  completed  initial  test  routes  without  reference  to 
feedback  mechanisms.  The  non-aviators  more  closely  approximated  the  level  of  terrain  navigation  experience  of  the 
target  user  group.  However,  the  target  user  group  would  have  an  average  of  90  fixed  wing  and  120  helicopter  hours. 
Additionally,  fixed  wing  training  involves  approximately  43  hours  in  various  simulators.  Helicopter  training  involves 
approximately  34  hours  in  the  simulator.  We  were  uncertain  if  this  would  impact  their  expectations  and  thus  adaptation 
time  (learning  curve)  to  the  interface. 

All  of  the  students  tested  at  HS-10  adapted  quickly  to  the  interface  and  were  able  to  control  motion  and  access 
feedback  mechanisms  easily.  As  predicted  by  the  usability  studies,  two  of  the  students  initially  expected  the  throttle 
lever  to  act  as  a  collective.  These  students  did  not  appear  to  have  any  more  difficulty  interacting  with  the  system  than 
the  other  students  after  the  first  few  minutes  of  exposure,  Based  on  observations  of  students  interacting  with  the  system, 
we  conclude  that  the  interface  was  in  fact  consistent  with  the  task  of  learning  to  interpret  contour  maps.  It  allowed 
students  to  experience  the  terrain  model  with  adequate  attention  to  resolving  the  egocentric  view  to  the  contour  map. 

The  training  system  appears  to  do  what  it  was  designed  for  -  provide  students  the  opportunity  to  improve 
their  ability  to  resolve  an  egocentric  view  with  a  contour  map  representation.  Based  on  observations  and  verbal 
protocol,  it  was  apparent  that  all  of  the  students  showed  at  least  incremental  improvement  in  this  skill.  This  will  require 
final  validation  after  their  training  flights  but  we  are  optimistic  that  we  will  see  at  least  moderate  gains  in  performance 
by  the  estimation  of  their  flight  instructors.  Additionally,  students  showed  a  wide  variance  in  both  initial  skill  level  and 
progress  made  during  the  training  sessions.  The  variance  in  both  initial  skill  level  and  progress  supports  the  concept  of 
asynchronous  access  and  easy  availability.  Clearly,  if  performance  during  the  training  sessions  can  be  shown  to 
correlate  to  performance  in  the  aircraft,  the  training  system  should  be  readily  available  to  all  pilots.  If  initial  skill  level 
and  rate  of  progress  vary,  students  should  be  able  to  access  the  system  as  many  times  as  they  need  to  for  how  ever  long 
than  feel  they  need  it. 

CONCLUSIONS 

This  experiment  is  incomplete  as  of  the  writing  of  this  paper.  Nevertheless,  we  feel  confident  that  the 
ability  to  practice  this  skill  in  a  system  such  as  this  one  will  prove  valuable  to  HS- 1 0.  Our  future  plans  include  an 
actual  field  test  of  a  system  to  allow  instructors  there  the  opportunity  to  see  if  it  has  a  place  in  the  curriculum. 
This  will  also  allow  us  the  opportunity  to  study  long  term  evaluation  periods  that  were  not  possible  in  this  first 
experiment.  The  steps  left  to  complete  this  goal  include  porting  the  system  to  the  Windows  NT  platform  and  a 
further  analysis  of  exactly  w  hat  effect  this  trainer  has  on  navigation  ability. 

Before  the  Navy  can  take  any  steps  toward  training  navigation  on  the  ground  in  lieu  of  in  the  air,  this 
effect  must  be  a  known  quantity.  However,  navigation  is  a  part  of  every  task  in  the  air,  so  this  trainer  would 
never  completely  replace  experience  in  the  aircraft.  The  primary  benefit  would  be  that  navigation  would  not 
explicitly  be  trained  in  the  air. 

Most  people  who  have  seen  the  system  point  out  its  potential  as  a  mission  rehearsal  system.  We  agree 
that  the  potential  exists  However,  there  is  a  hidden  danger  in  this  thinking.  It  is  easy  to  fall  into  the  trap  of 
training  only  routes  when  mission  rehearsal  system  are  used.  In  these  cases,  the  pilot  knows  how  to  get  to  the 
target  one  and  only  one  way.  If  problems  occur  during  the  mission  forcing  a  change  in  route,  the  pilot  is  actually 
worse  off  than  if  no  rehearsal  had  taken  place.  There  are  ways  around  this  pitfall  that  we  are  attempting  to 
identify  for  the  helicopter  community.  Again,  solutions  from  the  fixed  wing  community  may  not  apply  since 
flight  profiles  for  helicopter  missions  are  so  completely  different.  Nevertheless,  our  intention  is  to  continue  to 
learn  more  about  the  uniqueness  of  rotary  wing  aircraft  and  how  this  burgeoning  technology  can  be  brought  to 
bear  on  their  training  and  operational  problems. 
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INTRODUCTION 

It  is  likely  that  helmet-mounted  display  (HMD)  technology  will  be  fully  integrated  into  next-generation  tactical 
aircraft.  The  exact  utility  of  HMD  technology  can  not  yet  be  stated  as  confidently.  On  one  end  of  the  spectrum,  the  HMD 
may  simply  provide  an  aimsight  reticle  and  symbology  for  purposes  of  target  cueing  and  sensor  guidance.  The  other  end  of 
the  spectrum  sees  the  HMD  as  a  complete  replacement  for  the  head-up  display  (HUD)  and  the  primary  source  of  all  head- 
up  information.  Because  each  end  of  the  spectrum  has  its  respective  advantages  and  disadvantages,  the  evolution  of  the 
HMD  will  most  likely  lie  somewhere  between  these  extremes.  The  challenge  to  the  designer  tasked  with  determining  the 
information  content  and  utility  of  the  HMD  is  to  ensure  that  the  technology  provides  the  correct  information  at  the  correct 
time.  As  idealistic  as  this  statement  sounds,  it  is  in  essence  achievable  through  a  systematic  design  and  validated  evaluation 
approach.  This  paper  describes  what  we  hope  to  be  a  significant  contribution  toward  this  end. 

It  is  understood  that  the  primary  purpose  of  the  HMD  is  to  provide  target  cueing  information  to  the  pilot.  The  HMD 
and  the  information  associated  with  it  should  first  be  designed  to  get  the  pilot’s  eyes  on  a  target  and  lead  a  weapon  sensor  to 
a  point-of- interest.  In  parallel  arises  both  the  capability’  and  even  the  need  to  provide  other  types  of  information  via  the 
HMD.  For  instance,  a  unique  capability  of  the  HMD  is  to  present  ownship  status  information  (including  airspeed,  altitude, 
heading,  and  attitude)  to  the  pilot  regardless  of  head  location  or  movement  relative  to  the  aircraft  axes.  Figure  1  depicts  a 
simple  generic  example  of  how  this  ty  pe  of  information  may  be  formatted  w  ithin  the  HMD  field-of-view. 
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Figure  1 .  Generic  HMD  off-axis  ownship  information  symbology. 


Given  that  the  HMD  is  primarily  for  purposes  of  target  cueing,  significant  questions  remain:  what  is  the  effect  of 
including,  or  not  including,  ownship  information  to  the  pilot  when  the  pilot’s  visual  attention  is  concentrated  along  a  line- 
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of-sight  (LOS)  away  from  the  “head  forward”  display  areas  in  the  cockpit  (off-axis)?  If  in  fact  off-axis  target  cueing 
information  increases  targeting  situation  awareness  (SA),  what  is  happening  to  the  spatial  orientation  component  of  SA  in 
the  mean  time?  The  objective  of  this  paper  is  to  propose  an  evaluation  methodology  developed  to  address  these  important 
questions. 


BACKGROUND 

The  following  is  intended  to  provide  brief  background  into  past  research  which  has  suggested  SA  effects  related  to  the 
use  of  HMD  presented  information.  One  study  was  a  low  altitude  flight  task  which  included  a  with  and  without 
manipulation  of  off-axis  ownship  information.  Another  study  investigated  the  effects  of  various  types  of  HMD  presented 
target  cueing  information  during  air-to-air  engagements,  A  third  study,  also  air-to-air,  included  an  investigation  of  the 
HMD  information  content  level.  Included  were  HUD-only,  HMD  target  cueing  only,  and  HMD  target  cueing  with  off-axis 
information  levels. 

Low  altitude  ingress  study: 

One  of  the  first  studies  addressing  this  topic  included  a  simulated,  low-level,  high-speed,  airborne  surveillance  / 
reconnaissance  mission  (Osgood,  Geiselman,  and  Calhoun,  1991).  A  with  and  without  manipulation  of  a  simple  off- 
axis  ownship  information  display  was  performed.  Pilots  were  instructed  to  maintain  a  400  foot,  480  knot  flight  profile 
along  a  prescribed  heading.  Both  the  ground  and  the  threat  of  surface  to  air  missiles  were  included  as  adverse 
consequences  for  excess  altitude  deviations.  In  addition,  subjects  were  instructed  to  look  for  airborne  threats  (hostile 
aircraft)  and  take  evasive  action  if  fired  upon.  The  threats  were  designed  to  “pop-up”  behind  the  ownship  location. 
Each  trial  scenario  was  formed  of  a  search  phase  (looking  for  the  threat),  and  a  monitoring  phase  (watching  the  threat 
to  make  sure  it  did  not  take  hostile  action).  During  both  the  search  and  monitoring  tasks,  HMD  presented  ownship 
information  resulted  in  the  pilots  looking  farther  off-axis  for  a  longer  period  of  time.  This  effect  was  observed  in  the 
absence  of  a  flight  task  performance  difference.  Two  other  interesting  effects  were  found.  First,  during  the  142  trials 
of  the  experiment,  no  ground  strikes  occurred  when  HMD  ownship  information  was  available.  Without  the  HMD, 
five  ground  strikes  were  recorded.  Another  potential  insight  to  the  SA  benefits  of  the  technology  was  gleaned  by 
taking  a  snap-look  at  pilot  behavior  at  the  instant  a  significant  event  occurred:  specifically,  the  point  where  an  air-to- 
air  missile  was  launched  against  ownship.  As  seen  in  figure  2,  pilot  LOS  when  HMD  aided  tended  to  be  at  angles 
much  closer  to  the  location  of  the  threat  when  the  missile  was  launched.  In  fact,  during  trials  where  HMD  information 
was  not  available,  the  average  LOS  angle  actually  resulted  in  the  hostile  aircraft  location  being  beyond  the  reasonable 
FOV  of  the  pilot  (Figure  2). 


Figure  2  Snap-look  critical  event  performance. 


Air-to-air  target  cueing  study: 

Another  experiment  was  performed  to  investigate  the  effects  of  HMD  resident  target  location  information 
reference  frame  during  air-to-air  target  acquisition  and  intercept  tasks  (Geiselman  and  Osgood,  1995a).  The  objective 
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of  the  study  was  to  determine  if  off-axis  locator  information  should  indicate  the  position  of  the  target  relative  to  the 
nose  of  the  aircraft  (fly-to),  relative  to  the  nose  on  the  pilot’s  face  (look-to),  or  a  combination  (both).  Ownship  attitude 
information  was  not  presented  off-axis  for  this  study.  Eight  Air  Force  pilots  participated  in  the  study.  Each 
experimental  trial  included  target  search,  intercept,  tracking,  and  attack  components  within  an  air-to-air  tactical 
engagement  simulation.  During  the  target  intercept  task,  the  pilots  looked  farther  ofT-axis  longer  when  HMD  target 
location  information  was  available  This  relationship  was  consistent  throughout  ?he  tracking  and  attack  tasks  as  well 
The  pilots  looked  farther  ofT-axis  for  a  longer  period  of  time  due  to  the  target  location  information  presented  via  the 
HMD.  Although  not  supported  by  the  performance  data,  the  subjective  results  strongly  suggested  that  the  pilots 
favored  the  combination  (multiple  coordinate  reference  frame).  The  effect  of  this  behavior,  looking  farther  ofT-axis  for 
a  longer  period  of  time  without  the  assistance  of  ofT-axis  attitude  information,  on  the  ownship  awareness  component  of 
overall  SA  under  degraded  visual  conditions  has  yet  to  be  studied. 

Air-to-air  ownship  information  study: 

A  third  study  was  performed  to  explore  the  effect  of  various  levels  of  HMD  resident  information  on  performance 
during  air-to-air  target  acquisition  tasks  (Geiselman  and  Osgood,  1094).  The  information  levels  included  a  HUD-only 
baseline  (including  ownship  status  and  target  location  information),  a  HMD  with  target  location  information  only,  and 
a  HMD  with  both  target  location  (look-to  oriented)  and  ownship  status  information.  This  experiment  was  designed  as 
a  combination  of  the  previous  tw  o  with  the  exception  that  the  interest  area  was  limited  to  the  higher  altitude  air-to-air 
arena  (where  the  ground  was  not  an  immediate  threat).  For  this  study,  the  targeting  tasks  were  emphasized  over  the 
flight  tasks.  It  was  quite  possible  that  ownship  information  presented  within  the  HMD  would  result  in  more  clutter 
than  it  was  worth.  The  tasks  included  target  search,  acquisition,  and  attack  components.  Nine  Air  Force  pilots 
volunteered  to  participate  in  the  study.  Task  performance  and  pilot  head  behavior  was  recorded  and  analyzed.  In 
addition,  pilots  were  asked  to  record  subjective  judgment  ratings  of  situation  awareness  benefits  attributable  to  the 
information  condition  levels  and  ownship  information  formats.  The  following  operationally  oriented  definition  of  SA 
was  adopted  for  the  study:  “A  pilot’s  (or  aircrew’s)  continuous  perception  of  self  and  aircraft  in  relation  to  the 
dynamic  environment  of  flight,  threats,  and  mission,  and  the  capability  to  forecast,  then  execute  tasks  based  on  the 
perception.”  (Carroll,  1992).  The  subjective  SA  ratings  were  collected  using  the  SWORD  technique  (Vidulich,  Ward, 
and  Schueren,  1991)  modified  for  SA  rather  than  workload  reporting.  Subjects  were  asked  to  compare  all  possible 
pairs  of  the  information  levels  and  rate  the  extent  that  each  helped  to  enhance  SA  relative  to  the  others. 

Again,  during  the  search  task,  the  pilots  looked  farther  off-axis  for  a  longer  period  of  in  the  HMD  ownship  status 
information  condition  compared  to  either  the  HUD  only  or  HMD  target  location  information  levels.  This  of  course 
was  expected  because  no  target  location  information  was  displayed  until  a  radar  lock  was  established  and  the  attack 
phase  was  initiated.  During  the  attack  phase,  the  pilots  looked  farther  off-axis  fora  longer  period  of  time  with  both  the 
HMD  target  location  and  HMD  ownship  information  levels.  This  indicated  that  both  target  location  information  and 
ownship  status  information  results  in  pilots  looking  off-axis  According  to  the  SWORD  data,  the  pilots  rated  off-axis 
ownship  information  as  contributing  most  to  an  increase  in  SA.  HMD  presented  target  location  information  was 
second  and  HUD  only  information  was  last.  Again,  all  of  this  was  accomplished  in  the  absence  of  any  performance 
trade-off 

Research  conclusions: 

The  overall  conclusions  from  the  literature  are:  1)  HMD  presented  information,  compared  to  HUD-only 
information,  enabled  (or  compelled)  the  pilots  to  look  farther  off-axis  for  longer  periods  of  time  during  an  air-to-air 
task.  2)  The  effects  were  seen  independent  of  information  type  (target  information  vs.  ownship  attitude).  3)  Flight 
performance  was  not  affected  across  information  condition  levels  within  the  experimental  tasks.  4)  The  pilots 
preferred  that  off-axis  ownship  status  information  be  included  within  the  HMD  symbology  set  5)  As  described 
below,  specific  symbology  formats  intended  for  use  in  the  air-to-air  arena  should  be  designed  to  minimize  the  visual 
area  they  occupy 

In  general,  it  appears  that,  any  information  presented  on  the  HMD  compel  the  pilot  to  look  farther  off-axis  for  a 
longer  period  of  time.  Tin's  may  result  in  increased  targeting  SA  but  the  effect  on  the  spatial  orientation  component  of 
overall  SA  is  not  yet  known  It  is  important  that  we  avoid  an  increase  in  targeting  performance  only  to  accrue  a  cost 
elsewhere.  Though  a  good  systematic  design  process,  we  are  looking  for  a  solution  where  we  get  something  for 
nothing.  It  is  the  authors’  opinion  that  ownship  information  should  be  included  within  the  HMD  FOV,  provided  the 
information  is  well  designed  and  mechanized. 
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PROBLEM 

Intuitively,  it  makes  sense  that  ownship  information  is  included  in  the  ofT-axis  HMD  symbol  set.  But,  the 
symbology  projected  onto  the  HMD  occupies  valuable  visual  real-estate  For  this  reason,  incorporating  symbology  must  be 
justified  by  ar.  accompanying  performance  ivantage  or  critical  need.  Small  display  area  exacerbates  the  visual  occlusion 
problem  (the  instantaneous  field-of-view  of  operational  HMDs  is  typically  25-40  degrees  subtended  visual  angle).  In  the 
highly  visual  environment  of  air-to-air  combat,  it  is  of  critical  importance  to  maintain  an  unobstructed  central  FOV. 
Subjective  results  indicate  that  pilots  believe  more  information  is  better.  In  some  cases,  this  belief  is  not  reflected  in  the 
objective  findings.  This  is  evidence  that  a  good  performance-based  evaluation  methodology  is  needed  to  differentiate  “nice 
to  have”  from  information  that  will  produce  a  significant  advantage.  What  remains  is  to  ensure  that  the  proper  information 
in  the  proper  format  earns  it’s  way  onto  the  HMD.  The  main  reason  this  is  so  difficult  to  do  is  that  there  are  many  different 
forms  and  features  by  which  off-axis  ownship  information  can  be  conveyed  (see  below).  And,  there  is  no  comprehensive 
way  to  evaluate  these  variations  during  off-axis  targeting  representative  tasks. 

HMD  resident  ownship  information  variations: 

The  ownship  attitude  display  is  designed  to  convey  the  dynamic  relationship  between  ownship  axes  and  the  earth 
(heading  and  horizon).  The  design  and  evaluation  of  this  symbology  is  complicated  by  the  number  of  features 
available  to  the  designer  which  may,  in  some  interrelated  combination  of  form  and  function,  provide  optimized 
performance  under  specific  phase  of  flight  and  mission  conditions.  Form  is  the  shape  of  the  symbology,  how  it  is 
drawn,  while  function  is  how  the  symbology  is  dynamically  mechanized  and  referenced  to  the  earth.  Some  functions 
can  only  be  created  via  certain  forms  and  vice  versa.  Also,  because  of  the  availability  of  a  unique  helmet  coordinate 
reference  system,  attitude  information  can  be  presented  to  the  pilot  via  unconventional  formats.  For  this  reason,  a 
comprehensive  evaluation  methodology  should  incorporate  a  multi-task  phased  approach  designed  to  flesh  out  those 
features  of  the  HMD  presented  information  that  may  be  most  effective  across  applications. 

The  following  are  short  descriptions  of  some  of  the  options  available  to  the  designer:  included  are  symbol 
compression  ratio,  frame  of  reference,  observer  perspective,  and  axis  separation. 

Symbol  compression  ratio  (SCR): 

The  ratio  of  the  angle  represented  by  the  symbol  to  the  symbol's  subtended  visual  angle.  A  symbol  which 
represents  90  degrees  and  subtend  ,  five  degrees  has  a  SCR  of  18  1  (sy  mbol  to  world).  Compression  is  designed  into 
the  vertical  dimension  of  some  attitude  display  symbologies  in  order  to  reference  more  area  than  the  display  surface 
would  otherwise  be  capable.  SCR  is  a  critical  feature  and  will  influence  performance  in  a  flight-path  maintenance  task 
(Geiselman  and  Osgood,  1995b).  High  symbol  compression  results  in  formats  which  represent  large  angles,  and 
therefore  have  slow  rates-of-motion  when  compared  to  their  uncompressed  counterparts.  A  by-product  of 
compression  is  the  formation  of  an  artificial  horizon  which  does  not  consistently  conform  to  and  overlie  the  real 
horizon,  This  situation  has  traditionally  been  avoided  in  transparent  display  applications  such  as  the  HUD.  An  HMD 
application  affords  the  use  of  a  non-conformal  head-up  artificial  horizon  because  the  symbology  location  is  de-coupled 
from  the  aircraft  axes.  Geiselman  and  Osgood  (1995b)  compared  two  similar  compressed  attitude  display 
symbologies  to  an  uncompressed  HUD  format  The  more  highly  compressed  formats  resulted  in  the  best 
performance.  Presumably,  HUD  symbology  has  been  designed  under  the  assumption  that  low  compression  ratio  is 
effective  for  high  precision  tasks,  such  as  flight-path  maintenance,  and  a  high  SCR  is  preferable  for  low  precision 
tasks,  such  as  unusual  attitude  recovery. 

Frame  of  reference  (LOS  or  forward): 

Forward-referenced  information  on  the  HMD  depicts  a  view  as  if  looking  out  the  front  of  the  aircraft  regardless  of 
LOS  location,  orientation,  or  movement.  A  forward-referenced  attitude  display  will  only  update  due  to  aircraft 
maneuvering.  The  only  time  a  forward-referenced  attitude  display  can  appear  to  conform  to  the  movement  of  the 
outside  world  is  when  the  observer  is  looking  along  the  forward  axis  of  the  aircraft.  LOS-referenced  information  is  a 
depiction  of  a  view'  along  the  observer’s  LOS  In  this  way,  both  head  movements  (LOS  changes)  as  well  as  aircraft 
maneuvering  will  change  the  appearance  of  the  display.  A  LOS-referenced  display  can  be  drawn  to  conform  to  the 
movement  of  the  outside  world  relative  to  the  viewing  perspective  at  any  combination  of  azimuth  and  elevation  angles. 
A  LOS-referenced  approach  can  be  used  to  stabilize  information  with  the  earth  in  order  to  present  virtual  orientation 
cues.  Both  an  earth-stabilized  pathway  in  the  sky  and  a  conformal  horizon  reference  are  examples  of  this  approach 
The  effects  of  a  frame  of  reference  feature  manipulation  can  be  best  described  using  an  example  where  the  observer 
views  symbology  on  a  transparent  HMD  while  looking  at  the  outside  world  with  a  LOS  90  degrees  off  the  aircraft 
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flight  path  (fixed  wing).  For  a  forward-referenced  display,  during  a  rolling  maneuver,  the  artificial  horizon  component 
of  the  display  will  appear  to  be  completely  inconsistent  with  the  movement  of  the  natural  horizon.  For  a  LOS- 
referenced  display,  in  the  roll  input  condition,  the  movement  of  the  artificial  and  natural  horizon  are  consistent 
(conformal)  but  the  artificial  horizon  will  appear  to  indicate  a  pitch-like  vertica  ranslation  in  response  the  actual 
rotational  movement  of  the  aircraft  relative  to  its  flight  path.  Also,  because  a  LOS-referenced  display  will  respond  to 
head  movement,  changes  to  the  display  depiction  due  to  LOS  movement  may  be  confused  with  display  changes  due  to 
ownship  maneuvering. 

Observer  perspective  (inside-out  or  outside-in): 

This  concept  refers  to  the  perspective  from  which  the  observer  relates  ownship  attitude  to  the  earth.  The 
perspective  can  either  be  from  inside  ownship  looking  out,  or  from  outside  ownship  looking  in.  The  dynamics  of  the 
display  differ  dramatically  depending  on  which  orientation  is  incorporated  into  the  design.  With  a  consistent  inside- 
out  orientation,  a  moving  horizon  is  compared  to  a  fixed  ownship  symbol.  The  ownship  symbol  moves  against  a  fixed 
horizon  reference  in  a  pure  outside-in  display  (the  observer  interprets  the  display  as  if  standing  outside  of  the  aircraft). 

Axis  separation  (consistent  and  inconsistent): 

This  concept  refers  to  the  extent  the  above  features  are  applied  to  the  various  axes  of  the  ownship  attitude  display. 
A  consistent  feature  is  applied  the  same  way  across  the  display  axes  of  rotation.  Likewise,  features  can  be  applied 
inconsistently:  For  instance,  rotations  about  the  ownship  lateral  axis  (pitch)  may  be  depicted  as  outside-in  while 
rotation  about  the  longitudinal  axis  (roll)  is  represented  from  a  inside-out  perspective 

APPROACH 

Given  the  complexity  of  the  options  and  their  combinations,  a  common  method  to  evaluate  the  relative  benefit  (or 
cost)  afforded  by  the  various  HMD  ownship  attitude  symbology  approaches  is  called  for.  The  concepts  need  to  be 
evaluated  in  manner  which  affords  a  consistent  comparison  across  various  representative  tasks  that  are  characteristic 
(unique)  to  the  HMD  application. 

Proposed  Evaluation  Methodology: 

Held  constant  across  the  methodology  design  is  the  belief  that  the  primary  purpose  of  the  HMD  is  target  cueing. 
Therefore,  representative  evaluation  tasks  are  those  which  include  off-axis  target  searching,  designating,  and  tracking. 
The  objective  of  the  present  effort  is  to  develop  an  evaluation  methodology  that  can  be  used  to  compare  the  effect  of 
off-axis  ownship  information  on  these  tasks.  The  methodology  is  designed  to  be  flexible  so  future  candidate 
symbologies,  and  other  technologies,  such  as  multi-sensory  displays,  can  be  reliably  compared  to  prev  iously  collected 
data.  A  second  major  objective  is  to  develop  a  methodology  that  is  both  empirically  and  operationally  valid  A  final 
objective  is  to  produce  an  evaluation  methodology  that  is  experimentally  controlled  but  is  recognized  by  subject  matter 
experts  (SMEs)  as  operationally  relevant.  The  proposed  methodology  will  include  the  following  features* 

A  multi-phased  trial  approach  will  be  used  to  help  ensure  trial  continuity.  Each  trial  will  be  formed  of  separate 
phases  which  will  be  treated  and  analyzed  as  separate  tasks.  A  dual  task  paradigm  will  be  employed  with  off-axis 
targeting  (search,  location,  designation,  and  tracking)  primary  tasks.  The  secondary  tasks  include  flight  tasks  such  as 
attitude  maintenance,  maneuvering,  and  extreme  attitude  recovery.  Because  of  the  operational  nature  of  the  tasks,  at 
least  the  initial  evaluations  will  use  SMEs  as  experimental  subjects.  Independent  variable  manipulations  include  HMD 
ownship  symbology  format  type,  a  no  off-axis  ownship  information  baseline  condition,  and  natural  horizon  presence 
(on  or  off).  Measurement  metrics  will  include  task  performance,  subject  behavior  (head  movement),  and  subjective 
feedback.  Subjective  feedback  will  include  preference  questionnaires,  workload  estimates,  and  SA  ratings. 

The  methodology  is  being  designed  to  run  in  the  Air  Force  Research  Laboratory’s  Synthesized  Immersion 
Research  Environment  (SIRE)  facility  (Figure  3),  Wright-Patterson  AFB,  Ohio. 

SIRE  Facility: 

SIRE  provides  a  generic  single-seat  fighter  cockpit  with  out-the-window  visuals  presented  on  the  surface  of  a 
dome  projection  system  (40  feet  diameter).  The  projection  surface  is  1 50  degrees  horizontal  by  70  degrees  vertical 
field  of  view.  The  subject  will  be  seated  in  the  cockpit  located  20  feet  from  the  surface  of  the  dome  and  elevated  7.5 
feet  above  the  floor.  This  provides  a  vertical  separation  of  the  viewing  area  (50  degrees  above  the  horizon  and  20 
degrees  below  the  horizon  from  the  design  eye  point).  The  horizontal  separation  is  75  degrees  on  either  side  of  center. 
The  display  is  produced  by  six  highly  modified  BARCO  projectors  each  capable  of  a  resolution  of  1280  by  1024 
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pixels  at  60  Hz.  The  resolution  of  the  system  is  approximately  2  arc  minutes  per  pixel.  The  projectors  are  stacked  two 
high  and  three  across  to  cover  the  entire  viewing  area.  The  edges  of  the  viewing  area  generated  by  each  of  the 
projectors  are  blended  (two  to  four  degree  overlap)  in  order  to  create  a  uniform,  uninterrupted,  high-resolution  image 
across  the  entire  fi  .d  of  view.  The  image  generator  consists  of  a  Silicon  Graphics  Onyx  system  with  eight  processors, 
three  graphics  pipelines,  and  six  channels  of  video. 


fax- 


Figure  3  The  Synthesized  Immersion  Research  Environment  Facility. 


The  SIRE  cockpit  currently  includes  three  head-down  displays  and  one  head-up  display.  A  simulated  helmet- 
mounted  display  is  added  in  addition  to  the  standard  cockpit  displays.  Two  of  the  head-down  displays  are  6.6  inch 
square  color  displays  that  each  have  a  30  degrees  by  20  degrees  Field  of  view  These  displays  flank  a  center  display, 
and  are  mounted  one  on  the  left  and  the  other  on  the  right  side  of  the  crewstation.  For  all  phases  of  the  evaluation  trials, 
the  following  information  will  be  presented  on  the  head-down  display  surfaces  independent  of  the  HMD  off-axis 
symbology  mechanization:  (1)  integrated  ownship  primary  flight  information  suite  (attitude,  airspeed,  altitude, 
heading,  vertical  velocity,  and  angle  of  attack),  (2)  radar  display,  and  (3)  horizontal  situation  display.  The  HUD 
provides  collimated  monochrome  imagery  on  a  30  degrees  by  22  degrees  Field  of  view.  The  HUD  is  capable  of 
producing  computer-generated  imagery  (e.g.,  FL1R)  or  symbology.  For  all  phases  of  this  effort,  the  following 
information  will  be  included  as  a  HUD  presentation  independent  of  the  HMD  off-axis  attitude  symbology 
manipulation:  (a)  ownship  attitude,  airspeed,  altitude,  angle  of  attack,  and  magnetic  heading  and  (b)  aircraft  referenced 
(fly-to  oriented)  locator  line  indicating  the  ownship  pursuit  vector  to  the  target  location 

Head-Mounted  Display  (HMD): 

All  HMD  symbology  will  be  displayed  overlaid  upon  the  external  out  the  window  (OTW)  scene  presented  on  the 
SIRE  dome.  The  symbology  will  be  slaved  to  the  pilot’s  head  via  a  magnetic  position  tracker  attached  to  the  top  of  the 
helmet.  Thus,  the  symbology  will  always  be  centered  within  the  helmet  line  of  sight  and  steered  about  the  display  via 
head  movement.  The  following  components  will  be  included  in  the  HMD  symbol  set  independent  of  the  HMD  off- 
axis  attitude  symbology  manipulation:  (a)  Radar  LOS  reticule,  (b)  target  designator  (TD)  box,  (c)  HMD  LOS 
magnetic  heading  and  elevation,  and  (d)  look-to  oriented  head  referenced  locator  line  (to  indicate  the  azimuth  / 
elevation  vector  and  vector  length  to  the  target  location). 
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Procedure: 

Each  trial  will  be  comprised  of  three  distinct  task  phases  with  initial  conditions  to  test  the  efficacy  of  each 
symbolog}  ~et  in  terms  of  providing  attitude  information  to  the  pil  \  Specifically,  the  simulat  J  air  combat  scenario 
will  emphasize:  a  target  search  task,  a  dynamic  attitude  maintenance  task,  and  an  extreme  (unusual)  attitude  recovery 
task.  The  specifics  of  each  of  these  phases  will  be  detailed  in  the  individual  sections  to  follow  .  In  general,  each  pilot 
will  fly  an  equal  number  (6-8)  of  evaluation  trials  using  each  of  symbology  sets  of  interest.  The  symbology  set 
presentation  order  will  be  counterbalanced  between  pilots  to  guard  against  learning  and  order  effects  In  general,  for 
each  evaluation,  a  with  and  without  manipulation  of  HMD  off-axis  information  will  be  included  as  a  baseline 
performance  measure  Visual  condition  will  also  be  treated  as  a  general  independent  variable.  The  nighttime  lighting 
conditions  w  ill  be  used  across  trials  but  the  presence  of  the  v  isible  horizon  w  ill  be  manipulated 

The  following  paragraphs  present  the  basic  anatomy  of  a  single  experimental  trial  : 

Phase  one  —  Target  search  task: 

The  phase  one  objective  is  to  differentially  evaluate  how  well  the  candidate  off-axis  symbology  is  used  to 
maintain  flight  during  a  primary  task  (visual  search  for  a  point-of-interest) 

During  phase  one,  ownship  will  be  initialized  straight  and  level  at  a  medium  altitude  on  a  cardinal  heading  at  450 
kts.  The  primary  task  is  to  perform  a  visual  search  of  the  OTW  scene  for  a  target  that  will  randomly  pop-up  at  a  fixed 
azimuth  and  elevation  location  on  the  dome  display  surface.  Different  locations  will  be  used  from  trial  to  trial.  While 
performing  this  task,  the  pilots  are  to  maintain  their  prescribed  fight  parameters.  An  auditory  tone  will  command  left 
and  right  heading  changes  (90  degrees  each  time).  In  this  way,  the  fight  path  will  be  variable  but  task  performance 
will  be  independent  of  the  navigation  information  provided  to  the  subject.  Initial  target  location  relative  to  the  ownship 
location  will  be  designated  as  an  independent  variable.  A  hard  deck  altitude  will  also  be  assigned  for  the  scenario. 
The  hard  deck  altitude  will  be  approximately  10,000  feet  below  the  initialization  altitude.  Once  the  target  appears  (as  a 
small  designator  box),  the  pilot’s  task  is  to  slew’  a  head  stabilized  aimsight  reticle  over  the  target  location  as  quickly  as 
possible.  Once  the  target  is  inside  the  aimsight  reticle  (pilot  LOS  aligned  with  the  target  location),  the  pilot  presses  a 
button  and  the  trial  automatically  transitions  to  phase  two. 

Phase  one  dependent  variables:  primary  task  measures  will  include  target  search  variability,  reaction  time,  and 
trial  phase  time.  Head  movement  (angles,  duration,  and  rates)  will  enhance  the  performance  data.  Secondary  task 
performance  will  include  measures  of  airspeed,  altitude,  and  fight  path  (including  heading)  error  and  variability. 

Phase  two  —  Dynamic  attitude  maintenance  task: 

The  phase  two  objective  is  to  differentially  evaluate  how’  well  the  candidate  off-axis  symbology  is  used  to 
maintain  maneuvering  fight  while  maximizing  LOS  time  on  a  point-of-interest  located  off-axis. 

During  phase  two,  the  aimsight  reticle  that  was  head  fixed  during  phase  one  becomes  aircraft  fixed  at  the  azimuth 
and  elevation  angle  where  the  button  press  occurred.  Now,  the  target  location  that  was  aircraft  fixed  begins  to 
maneuver  relative  to  ownship  We  essentially  start  to  drive  the  TD  box  along  a  pre-recorded  flight  path  at  rates  that 
are  representative  of  high-performance  aircraft  maneuvering.  The  pilot’s  task  is  to  try  and  maintain  the  TD  box 
location  within  the  aircraft-fixed  aimsight  reticle  by  maneuvering  ownship  simultaneously  about  the  roll  and  pitch 
axes.  This  is  basically  a  formation  fight  task.  If  followed  well,  the  movement  of  the  TD  box  will  eventually  fy 
ownship  into  an  extreme  attitude  (both  isolated  and  combinations  of  extreme  pitch  (<60  degrees)  and  roll  (<90 
degrees)).  The  TD  box  will  then  disappear  to  simulate  a  lost  lead  or  lost  target  condition.  This  event  completes  phase 
two  and  initiates  phase  three. 

Also  during  phase  two,  some  catch  trials  will  be  included  in  order  to  test  specific  SA  components.  In  these  cases, 
the  target  will  maneuver  ownship  into  a  situation  that,  if  continued,  will  result  in  a  hard  deck  violation.  Another 
variation  of  this  may  result  in  dangerously  low  airspeed.  In  these  cases,  the  pilot’s  proper  response  is  to  break  off  the 
tracking  task  to  keep  from  violating  the  hard  deck  or  avoid  departing  controlled  fight.  These  trials  should  address 
some  target  fixations  issues. 

Phase  two  dependent  variables:  The  primary  task  measure  of  interest  will  be  target  tracking  performance.  Head 
movement  (angles,  duration,  and  rates)  will  enhance  the  performance  data.  Secondary  task  performance  will  include 
measures  of  ownship  energy  and  maneuvering  performance. 
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Phase  three  —  Extreme  attitude  reeovery  task: 

The  phase  three  objective  is  to  differentially  evaluate  how  well  the  candidate  off-axis  symbology  is  used  to 
recover  from  extreme  attitudes  while  maximizing  LOS  time  off-axis  without  an  external  orientation  cue. 

Once  the  target  disappears,  phase  three  begins  in  the  lost  target  condition.  A  target  search  task  immediately 
resumes  (similar  to  phase  one)  w'hile  the  pilot  is  tasked  to  fly  ownship  from  the  extreme  attitude  incurred  during  phase 
two,  to  a  new  commanded  altitude  and  heading.  This  phase  is  formed  of  two  sub  tasks.  During  the  initial  recovery, 
the  subject  will  perform  the  target  search  task  Once  the  target  is  located  (later  in  the  recovery  process),  the  task  will 
include  maintaining  LOS  on  the  target  location  as  much  as  possible  while  refining  the  recovery'.  This  is  an  attempt  to 
collect  unusual  attitude-like  performance  data  while  maintaining  a  high  level  of  task  realism  This  phase,  and  the 
experimental  trial,  will  end  when  the  subject  flies  ownship  to  within  a  specified  combination  of  heading,  altitude, 
attitude,  and  airspeed  for  a  specified  period  of  time. 

Phase  three  dependent  variables:  The  primary  task  measure  of  interest  will  be  target  search  and  tracking 
performance.  Head  movement  (angles,  duration,  and  rates)  will  enhance  the  performance  data.  Secondary  task 
performance  will  include  measures  of  reaction  time,  initial  input  error,  control  reversal,  recovery  time,  recovery  error, 
and  ownship  energy. 

Subjective  feedback: 

Questionnaires  will  be  included  to  address  pilot  preference  and  suggestions.  Included  in  the  subjective  feedback 
will  be  measures  of  both  workload  and  SA.  The  SWORD  technique  is  being  considered  as  the  workload  metric  while 
Cognitive  Compatibility-Situational  Awareness  Rating  Technique  (CC-SART)  is  being  considered  as  the  subjective 
SA  measurement  technique.  Performance  data  will  be  compared  to  the  subjective  findings  as  a  validity  check. 

DISCUSSION 

Presently,  the  laboratory  is  completing  the  software  development  for  this  project.  The  first  data  collection  activity  will 
include  two  goals:  First,  we  hope  to  validate  the  methodology  as  well  as  collect  SME  input  for  improvement  The  second 
goal  is  to  begin  evaluation  data  collection  for  candidate  symbol  sets.  Future  plans  for  the  project  include  broadening  the 
scope  of  the  test  functionality  to  include  more  application  general izability.  It  is  likely  that  the  methodology  can  be 
expanded  to  address  both  off-axis  ownship  information  issues  as  well  the  combined  use  of  various  target  cueing  symbology 
candidates.  Eventually  this  approach  can  grow  to  be  a  valid  and  reliable  method  by  which  future  HMD  symbology  can  be 
efficiently  compared  From  this  comparison,  performance-based  design  decisions  can  be  confidently  quantified 
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ABSTRACT 

In  this  effort,  we  describe  a  theoretically  based  methodology  for  assessing  team  SA  named  SAL1ANT 
(Situational  Awareness  Linked  Instances  Adapted  to  Novel  Tasks).  SAL1ANT  results  in  an  event-based 
scenario  and  measurement  tool  that  provides  multiple  opportunities  to  evaluate  teams  based  on  behaviors 
associated  with  team  SA.  In  addition,  we  report  an  initial  validation  study  conducted  to  evaluate  the 
psychometric  properties  of  SALiANT  (i.e.,  reliability  and  validity).  Results  indicated  that  team  SA  ratings 
derived  from  SALIANT  were  related  to  team  communication  and  performance  ratings;  however,  no  relationship 
was  found  between  team  SA  ratings  and  indices  of  team  shared  mental  models.  Based  on  these  results,  we 
discuss  lessons  learned  in  designing  and  validating  team  SA  measurement  tools. 

INTRODUCTION 

The  success  of  today’s  military  operations  depends  heavily  on  the  effective  performance  of  teams. 
Given  their  crucial  role  in  these  complex  settings,  it  is  important  to  understand  what  determines  such  effective 
performance.  One  factor  often  reported  to  affect  a  team’s  functioning  is  situational  awareness  (SA)  (Prince  & 
Salas,  1993).  SA  is  important  for  teams  because  it  allows  team  members  to  be  attentive  to  changes  in  the 
environment  and  anticipate  the  consequences  of  these  variations  (Stout,  Cannon-Bowers,  Sc  Salas,  1996).  A 
breakdown  in  a  team’s  process  of  acquiring  and  maintaining  SA  often  leads  to  disastrous  consequences  In  fact, 
aviation  mishap  reports  often  associate  a  large  percentage  of  human  error  related  accidents  in  military  settings  to 
problems  with  SA  (Hartel,  Smith,  &  Prince,  1991).  Such  evidence  reflects  the  importance  to  learn  about  the 
components  that  ensure  the  effective  achievement  of  SA  in  teams.  Moreover,  increasing  our  understanding  of 
team  SA  will  facilitate  the  development  of  strategies  such  as  training  programs  and  advanced  displays  designed 
to  deter  team  failures  associated  with  SA  deficiencies.  These  strategies,  in  turn,  need  to  be  evaluated  to 
determine  their  impact  on  enhancing  a  team’s  SA.  To  accomplish  this,  it  is  necessary  to  develop  valid  and 
reliable  instruments  that  can  provide  an  index  of  a  team’s  SA  level.  Unfortunately,  few  measures  of  team  SA 
exist  (Mufiiz,  Stout,  Bowers,  &  Salas,  1998;  Stout  et  al.,  1996).  To  address  the  current  needs  in  team  SA 
measurement,  this  effort  sought  to  develop  and  validate  a  theoretically  based  methodology  for  assessing  SA  in  a 
team  context.  Before  we  elaborate  on  details  about  this  methodology,  some  background  information  that  was 
used  as  a  foundation  for  this  research  will  be  reviewed,  including  SA,  SA  measures,  and  team  SA. 

Situational  Awareness .  Previous  research  that  focuses  on  SA  has  contributed  primarily  to  our 
understanding  of  how  individual  team  members  acquire  SA.  One  of  the  most  commonly  accepted  theoretical 
frameworks  that  centers  on  explaining  individual  SA  is  provided  by  Endsley  (1988;  1995)  In  this  theory,  she 
defines  SA  as  “the  perception  of  elements  in  the  environment  within  a  volume  of  time  and  space  [level  I],  the 
comprehension  of  their  meaning  [level  2],  and  the  projection  of  their  status  in  the  near  future  [level  3]” 
(Endsley,  1988;  p  7).  These  components  are  defined  in  three  levels,  and  they  are  proposed  to  be  hierarchical  in 
nature  for  the  achievement  of  SA. 

The  components  derived  by  Endsley  have  some  implications  for  measurement,  More  specifically,  she 
derived  a  measurement  methodology  which  attempts  to  capture  an  individual’s  SA  at  different  levels  with  the 
third  level  suggesting  to  be  the  highest  SA  that  could  be  achieved  {Situation  Awareness  Global  Assessment 
Technique  (SAGAT),  Endsley,  1988}.  In  addition  to  the  SAGAT  technique,  a  number  of  measurement 
methodologies  are  available  if  the  goal  is  to  assess  the  level  of  SA  acquired  by  individual  team  members  (e.g., 
memory  probes,  Wellens,  1993;  subjective  ratings,  Arbak,  Schwartz,  Si  Kuperman,  1987)  A  review  of  these 
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measures  is  beyond  the  scope  of  this  effort  However,  such  measures  have  been  criticized  on  a  host  of  grounds, 
primarily  because  of  the  lack  of  empirical  data  that  demonstrates  their  psychometric  value  (i.e.,  reliability  and 
validity)  (Fracker,  1991).  More  importantly,  these  measures  are  insufficient,  if  one  is  concerned  with  capturing 
the  team  clement  of  situation  awareness.  This  is  because  team  SA  is  more  than  simply  combining  individual 
team  members’  situation  awareness  (Schwartz,  1990).  In  the  next  section,  we  briefly  address  the  team 
component  of  SA  and  its  impact  on  measuring  it. 

Team  Situational  Awareness.  Team  tasks  that  are  complex,  dynamic,  and  information  rich,  often 
require  that  every  team  member  obtain  a  certain  level  of  SA  based  on  the  assessment  of  cues  and  events  present 
in  the  environment  In  turn,  each  team  member’s  SA  is  modified  as  information  is  exchanged  between  members 
who  may  have  observed  a  cue  or  event  that  could  be  vital  for  the  effective  performance  of  the  team  Attaining 
SA  at  the  team  level  is  a  complicated  process  given  that  there  are  a  number  of  interactive  behaviors  (e.g., 
information  sharing,  coordination)  that  play  a  crucial  role  in  the  achievement  and  maintenance  of  SA  for  al] 
team  members  (Stout  et  al.,  1996;  Schwartz,  1990).  In  fact,  a  review  of  literature  on  team  situation  awareness 
revealed  that  at  least  two  components  are  crucial  for  its  achievement*  (a)  individual  SA,  and  (b)  team  processes 
that  help  build  the  SA  of  a  team  (Stout  et  al.,  1996;  Salas,  Prince,  Baker,  &  Shrestha,  1995). 

The  team  element  of  SA  has  been  addressed  by  a  number  of  researchers  at  the  cognitive  level  (Stout  et 
al.,  1996),  and  behavioral  level  (Stout  et  al.,  1996;  Prince  &  Salas,  1993,  Mosier  &  Chidester,  1991;  Bunecke, 
Povenmire,  Rockway,  &  Patton,  1990;  Schwartz,  1990;  Leedom,  1990;  Foushee,  1984).  In  addition,  there  have 
been  numerous  studies  which  have  investigated  constructs  theorized  to  be  related  to  team  SA  (e.g., 
communications ,  Schwartz,  1990;  Palmer,  1990;  Wellens,  1993;  Bowers,  Braun,  &  Kline,  1994;  Mufliz,  Stout,  & 
Salas,  1996,  shared  mental  models.  Stout,  1995;  Stout  et  al.,  1996,  and  performance ,  Bowers,  Barnett,  Weaver, 
&  Stout,  1998). 

The  available  literature  on  team  SA  delineates  components  that  can  be  used  to  derive  measurement 
methodologies  for  evaluating  SA  in  teams.  Measures  of  team  SA  should  consider  both  cognitive  and  behavioral 
processes  that  indicate  its  presence  and  absence.  While  both  of  these  team  SA  components  are  equally  important 
(i.e.,  behaviors  and  cognition),  at  this  point,  we  focus  on  the  behavioral  component  of  team  SA  and  its 
implications  for  measurement. 


A  METHODOLOGY  FOR  MEASURING  TEAM  SA 

Previous  team  research  has  created  event-based  measurement  methodologies  for  evaluating  the 
performance  of  av  iation  teams.  (For  further  details  on  event-based  scenario  measurement  methodologies  please 
refer  to  Fowkles  et  al.,  1994).  These  methodologies  are  believed  to  have  the  potential  for  measuring  SA  in 
teams  (Mufiiz  et  al.,  1998;  Bowers  et  al.,  1998.)  Thus,  we  build  upon  this  work  to  create  an  event-based 
approach  for  measuring  team  SA. 

The  methodology  that  provides  opportunities  to  capture  team  SA  has  been  termed  SAL1ANT 
(Situational  Awareness  Linked  Instances  Adapted  to  Novel  Tasks)  (Mufiiz  et  al.,  1998).  By  implementing  this 
methodology  one  can  derive  (a)  theoretically  based  team  SA  behaviors,  (b)  scenario  events  which  provide  teams 
with  opportunities  to  manifest  SA  behaviors,  and  (c)  a  structured  behavioral  checklist  to  determine  a  team’s 
level  of  SA.  The  SAL1ANT  methodology  is  comprised  of  five  phases.  Each  phase  is  necessary  to  ensure  that 
team  SA  is  assessed  reliably  and  objectively. 

PHASE  1.  Delineation  of  Behaviors  Theoretically  Linked  to  Team  SA.  Twenty-one  (21)  generic 
behaviors  theorized  to  manifest  a  team’s  level  of  SA  were  identified  via  literature  reviews  and  investigations 
with  a  focus  in  aviation  (e.g.,  reports  of  aviation  mishaps,  observations  of  aviation  teams,  and  aviator’s 
responses  to  critical  incident  reviews).  Although  a  comprehensive  review  of  this  literature  is  beyond  the  scope 
of  this  effort,  a  summary  list  is  provided  in  Table  1.  These  generic  behaviors  were  used  as  the  foundation  for  the 
next  four  phases. 
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Table  1 .  Generic  Behavioral  indicators  of  team  situation  awareness  (Mufliz  et  al.,  1998). 


Demonstrated  Awareness  of  Surrounding  Environment 
•Monitored  environment  for  changes,  trends,  abnormal  conditions  (Prince  1 998) 
•Demonstrated  awareness  of  where  he/she  was  (Bunecke  et  al., *  *990) 

Recognized  Problems 

•Reported  problems  (Prince  &  Salas,  1993;  Foushee,  1984) 

•Located  potential  sources  of  problem  (Prince  &  Salas,  1993) 

•  Demonstrated  knowledge  of  problem  consequences  (Prince,  1998) 

•Resolved  discrepancies  (Schwartz,  1990) 

•Noted  deviations  (Prince  &  Salas,  1993) 

Anticipated  a  Need  for  Action 

•Recognized  a  need  for  action  (Prince,  1998;  Prince  &  Salas,  1993;  Foushee,  1984) 
•Anticipated  consequences  of  actions  and  decisions  (Prince,  1 998) 

•  Informed  others  of  actions  taken  (Leedom,  1990) 

•Monitored  actions  (self  &  others)  (Prince  &  Salas,  1993) 

Demonstrated  Knowledge  of  Tasks 
•Demonstrated  knowledge  of  tasks  (Schwartz,  1990) 

•Exhibited  skilled  time  sharing  attention  among  tasks  (Schwartz,  1 990) 

•Monitored  workload  (self  &  others)  (Prince,  1998) 

•Shared  workload  within  station  (Bunecke  et  al.,  1990) 

•Answered  questions  promptly  (Prince,  1998) 

Demonstrated  Awareness  of  Information 

•Communicated  important  information  (Bunecke  et  al.,  1990) 

•Confirmed  information  when  possible  (Bunecke  et  al.,  1990;  Leedom,  1990) 
•Challenged  information  when  doubtful  (Prince  &  Salas,  1993;  Leedom,  1990;  Bunecke  et  al.,  1990) 
•Re-checked  old  information  (Mosier&  Chidester,  1991) 

•Provided  information  in  advance  (Prince  &  Salas,  1993;  Schwartz,  1990) 

•Obtained  information  of  what  is  happening  (Foushee,  1984) 

•  Demonstrated  understanding  of  complex  relationships  (Bunecke  et  al.,  1990,  Schwartz,  19 90) 
•Briefed  status  frequently  (Prince,  1998;  Schwartz,  1990) 


PHASE  11.  Development  of  Scenario  Events.  Aviation  related  scenario  events  were  developed  to 
provide  teams  with  multiple  opportunities  to  demonstrate  team  SA  behaviors.  These  events  were  developed 
based  on:  (a)  input  from  subject  matter  experts  (SMEs);  (b)  the  level  of  simulation  in  which  the  scenario  is 
performed,  and  (c)  an  analysis  conducted  to  ensure  the  scenario  is  complex,  dynamic  and  demands  team 
interactions.  Each  of  these  events  was  designed  to  elicit  specific  behavioral  indicators  of  team  SA.  The 
requirement  of  linking  generic  indicators  to  specific  scenario  events  is  a  key  component  of  SALIANT  because  it 
provides  opportunity  to  delineate  specific  manifestations  of  team  SA  from  which  one  can  infer  the  level  of  SA 
acquired  by  a  team. 

PHASE  111  Identification  of  Specific  Observable  Responses.  Researchers  and  SMEs  identified  a 
number  of  specific  observable  responses  that  were  expected  to  be  exhibited  by  a  team  exposed  to  the  scenario 
These  responses  were  then  clustered  into  one  of  the  five  flight  factors  identified  by  Wagner  and  Simon  (1990) 
(i.e.,  mission  objectives,  orientation  in  space,  external  support,  equipment  status,  and  personal  capabilities). 
These  factors  were  identified  as  critical  for  aviation  teams  to  attend  and  maintain  effective  SA.  The  objective  of 
clustering  specific  responses  into  these  categories  is  to  ensure  their  task  relevancy  to  the  aviation  domain.  In 
addition,  in  each  of  these  categories,  we  included  specific  behaviors  that  take  in  consideration  the  anticipatory 
component  to  acquire  high  levels  of  SA  (Stout  et  al.,  1996,  Endsley,  1995).  The  identification  of  these 
anticipatory  behaviors  is  what  differentiates  these  behaviors  from  other  team  processes  associated  with 
teamwork. 

PHASE  IV.  Development  of  a  Script.  After  all  the  acceptable  responses  for  each  scenario  segment 
were  delineated,  a  scenario  script  was  developed,  The  objective  of  the  script  was  to  ensure  that  teams  were 
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presented  with  similar  information  and  situations.  In  addition,  the  script  contained  information  on  how  the 
experimenter  should  respond  if  anticipated  problems  arose. 

PHASE  V.  Development  of  an  Observation  Form.  The  work  accomplished  in  Phases  I  IV  was  used 
to  develop  a  structured  observation  form  rate  teams  on  the  number  of  specific  observable  behaviors  exhibited. 
The  form  was  organized  into  4  columns  that  specify  scenario  segments,  specific  observable  responses,  specific 
team  SA  behaviors  being  measured.  The  specific  observable  responses  were  coded  using  a  dichotomous  system 
(i.e.,  present  or  absent). 

In  summary,  the  SAL1ANT  methodology  was  developed  to  attempt  to  capture  team  manifestations  of 
SA  based  on  theoretically  derived  behaviors.  (A  summary’  of  this  methodology  is  depicted  in  Figure  1),  This 
methodology  was  used  as  a  basis  to  conduct  an  initial  study  to:  (a)  evaluate  the  extent  to  which  team  SA  can  be 
measured  reliably  by  using  a  SAL1ANT  based  tool,  and  (b)  determine  the  relationship  of  a  SALIANT  based 
measure  to  constructs  theorized  to  be  related  to  team  SA  (i.e.,  communication,  shared  mental  models,  and 
performance).  The  goal  of  this  effort  was  to  determine  the  effectiveness  of  a  SALIANT  based  methodology  at 
assessing  team  SA. 
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Figure  1.  SALIANT  Methodology 


SALIANT  VALIDATION  STUDY 


Methodology 

Participants.  Twenty  undergraduate  students  from  a  Southeastern  university  participated  in  a  4-hour 
experiment.  These  students  received  extra  credit  for  one  of  their  psychology  classes  for  participating  in  this  stud} . 

Materials  and  Equipment. 

Simulator.  A  tabletop,  commercially  available,  software  simulation  named  Comanche  was  employed  in  this 
investigation.  This  software  was  created  to  simulate  the  characteristics  of  a  Reconnaissance/Attack  Army  helicopter 
(Boeing  Sikorsky  RAH-66  Comanche).  The  software  presents  an  instrument  panel  and  external  visual  aids  on  a 
computer  monitor.  This  software  was  operated  in  a  486  personal  computer.  Other  materials/peripherals  were  used  to 
simulate  communication  (e.g.,  live  and  pre-recorded  messages  occurring  outside  the  cockpit),  and  navigation 
equipment  (e.g.,  headphones,  a  joystick,  a  clock,  and  a  map  with  pre-determ ined  routes). 
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Evaluation  Scenario  Using  the  SALIANT  methodology,  an  event-based  scenario  was  developed  to  elicit 
team  SA  behaviors.  In  this  scenario,  participants  were  required  to  role-play  an  Army  attack  helicopter  aircrew  from  an 
air  cavalry  unit.  The  scenario  required  participants  to  fly  in  enemy  territory,  report  the  location  of  targets  to  an  Air 
Mission  Commander  (played  by  an  experimenter),  and  destroy  the  targets.  To  accomplish  these  tasks,  participants 
were  required  to  perform  in  2-person  teams.  One  participant  played  the  role  of  the  pilot,  who  was  responsible  for  flying 
the  aircraft  and  communicating  over  the  headsets  with  other  aviation  units.  The  other  participant  played  the  role  of  the 
co-pilot,  who  was  responsible  for  providing  directions  to  the  pilot  and  destroying  targets.  As  previously  discussed,  a 
script  was  developed  for  the  experimenter  which  specified  when  events  were  introduced,  what  to  verbalize  during  the 
scenario,  and  how  to  respond  to  anticipated  problems. 

Data  Acquisition.  The  teams’  performance  was  recorded  by  two  methods.  A  video  camera  was  used  to 
capture  a  view  of  the  teams  from  the  back  of  the  experimental  room.  A  second  device  called  “Tvator"  ™  was  used  to 
convert  computer  signals  from  the  simulation  into  a  television  signal,  which  was  then  recorded  on  videotape.  In 
addition,  the  audio  signal  from  the  headphones  was  transferred  to  the  audio  track  of  both  types  of  recordings. 

Measures. 

Team  SA.  Two  trained  raters,  using  a  structured  observation  form  derived  from  the  SALIANT  methodology, 
assessed  the  videotapes  depicting  each  teams  performance  from  the  back  of  the  room.  Teams  received  a  point  for  each 
behavior  exhibited  Thus,  the  total  number  of  behaviors  exhibited  was  considered  to  be  the  level  of  team  SA  acquired 
by  a  team 

Communications  Videotapes  were  also  used  to  code  the  frequency  of  communications  exhibited  by  the 
teams.  A  different  pair  of  raters  documented  the  communication  frequencies  exhibited  by  the  teams,  and  these 
frequencies  were  coded  into  nine  categories:  commands,  inquiries,  statement  of  intent,  suggestions,  acknowledgments, 
replies,  non-task  related,  and  uncodable  (Bowers  et  al.,  1994;  Bowers,  Deaton,  Oser,  Prince,  &  Kolb,  1995;  Oser, 
Prince,  Morgan,  &  Simpson,  1991).  The  number  of  communications  was  added  to  obtain  an  overall  communication 
score. 

Shared  Mental  Models .  A  questionnaire  was  provided  to  participants  after  completing  the  scenario.  In  this 
questionnaire,  participants  were  asked  to  give  likelihood  in  which  they  expected  themselves  and  their  teammates  to 
exhibit  certain  behaviors  in  a  subsequent  mission.  The  teams’  agreement  on  this  questionnaire  was  calculated  by  using 
the  Lawlis  and  Lu  (1972)  agreement  coefficient.  These  coefficients  were  considered  to  be  an  index  of  the  teams  shared 
mental  models  with  regards  to  team  expectations  about  the  task  and  team  member  roles  (Blickensderfer,  1 996). 

Performance.  The  team’s  overall  performance  was  assessed  by  the  number  of  targets  destroyed  in  their 

mission. 

Results . 

Data  were  examined  to  determine  the  viability  of  SALIANT  for  team  SA  measurement  Specifically, 
data  were  examined  for  evidence  of  reliability  and  validity. 

Inter-observer  Reliability . 

Team  SA  Ratings  Analyses  were  performed  to  determine  the  level  of  agreement  between  the  two  observers 
that  used  a  checklist  derived  using  the  SALIANT  methodology  to  assess  the  SA  of  teams.  Significant 
correlations  were  found  between  the  team  SA  ratings  made  by  the  two  observers  (r  =  .94,  p  <  .05).  Thus,  the 
strong  relationship  found  between  the  ratings  made  by  the  two  observers  suggests  that  they  were  agreeing 
substantially  in  the  ratings  of  team  SA  behaviors  by  using  a  SALIANT  derived  measure. 

Communications  Ratings.  Analyses  were  performed  to  determine  the  level  of  agreement  between  the  two 
observers  that  used  the  communication  rating  form  to  determine  the  total  number  of  communication 
frequencies  engaged  in  by  each  team.  Significant  correlations  were  found  between  the  communication  ratings 
made  by  the  two  observers  (r  =  .98,  p  <  .05).  Thus,  the  strong  relationship  between  the  communication 
ratings  made  by  the  two  observers  suggests  that  they  were  consistent  in  using  the  communication  frequency 
form. 

Validity.  Analyses  were  performed  to  assess  the  relationship  between  teams’  SA  scores  and  total  number  of 
communication  frequencies,  index  of  shared  mental  models,  and  performance.  Significant  correlations  were 
found  between  team  SA  scores  and  total  number  of  communication  frequencies  (r  =  .74,  p  <  .05),  and 
between  team  SA  scores  and  performance  (r  =  .63,  p  <  .05).  Non  significant  correlations  were  found  between 
team  SA  scores  and  teams’  shared  mental  model  indices  (r  =  -  .04,  p  >  .05).  These  results  suggest  that  the 
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SALlANT-derived  measure  might  be  a  good  indicator  of  the  relationship  of  team  SA  and  communication  and 
performance,  but  not  for  shared  mental  models. 

DISCUSSION 

Findings  in  this  effort  are  significant  because  they  begin  to  advance  our  understanding  of  situation  awareness  at 
the  team  level.  Further,  the  current  effort  begins  to  address  recent  measurement  problems  raised  within  the  SA 
literature.  Specifically,  many  researchers  have  noted  the  psychometric  deficiencies  in  available  SA  measures  (i.e.,  little 
reliability  and  validity  evidence),  and  the  lack  of  methods  and  measures  that  capture  the  team  element  of  SA.  The 
development  of  these  measures  is  crucial  to  advance  our  understanding  of  factors  affecting  SA  in  teams,  as  well  as  to 
evaluate  strategies  developed  to  augment  SA  in  teams.  Given  the  importance  of  measurement  for  both  researchers  and 
operators,  the  current  effort  sought  to  accomplish  two  goals.  One  goal  was  to  document  a  theoretically  based 
methodology  that  aims  at  assessing  SA  in  teams.  The  second  objective  was  to  report  evidence  from  an  initial  validation 
study  to  assess  the  psychometric  properties  of  this  methodology. 

Findings  indicate  that  there  was  a  significant  agreement  among  raters  that  used  an  observation  checklist  derived 
from  SALIANT,  which  provides  evidence  of  how  reliably  we  can  assess  SA  across  teams.  In  addition,  results  indicated 
there  was  a  high  relationship  between  team  SA  scores  and  communication  ratings.  Also,  there  was  a  relationship 
between  team  SA  scores  and  performance  ratings.  These  findings  show  some  promise  that  the  scale  derived  from 
SALIANT  might  be  useful  for  predicting  these  two  constructs  after  further  analysis.  Our  results  also  suggested  a  non¬ 
significant  relationship  between  team  SA  scores  and  teams’  shared  mental  model  indices. 

There  are  several  hypotheses  to  explain  the  non-significant  correlation.  First,  the  shared  mental  model 
questionnaires  were  completed  at  the  end  of  the  experiment  and  perhaps  the  length  of  the  experiment  resulted  in 
fatigued  participants.  Second,  the  adaptation  of  these  questionnaires  to  another  team  setting  and  team  task  needs  further 
validation  itself.  Finally,  given  that  shared  mental  models  is  a  cognitive  based  construct,  there  is  possibly  the  need  to 
use  a  combination  of  measures  that  tap  various  dimensions  of  this  construct 

Based  on  these  results,  several  lessons  learned  can  be  derived  First,  while  the  importance  of  cognitive  processes 
for  acquiring  team  SA  is  acknowledged,  this  issue  was  not  addressed  with  the  SALIANT  methodology.  Cognitive 
based  measures  are  recommended  to  complement  SALIANT.  Second,  the  21  behaviors  used  as  a  foundation  for 
SALIANT  are  considered  to  be  manifestations  of  high  team  SA  More  work  needs  to  be  conducted  to  determine 
behavioral  indicators  of  low  team  SA  to  develop  a  measurement  methodology  to  supplement  SALIANT.  Third,  event- 
based  scenario  development  requires  individuals  with  extensive  experience  with  the  tasks  and  methodology  to  ensure 
the  scenario  is  properly  constructed  (i.e.,  elicit  natural  responses  from  the  teams).  Finally,  the  identification  of  specific 
responses  is  a  labor-intensive  effort.  In  some  occasions,  the  teams  may  not  exhibit  specific  responses,  and  at  other 
times  teams  may  exhibit  unanticipated  responses. 

On  a  more  positive  side,  the  current  effort  found  evidence  that  SALIANT  has  potential  for  measuring  SA  in  a 
team  context.  Given  the  documented  importance  of  SA  in  team  performance,  it  is  important  to  identify  what  factors 
modify  its  achievement  These  factors,  in  turn,  need  to  be  addressed  in  strategies  developed  to  augment  SA  in  teams. 
Thus,  more  studies  are  encouraged  to  be  conducted  that  continue  to  test  and  refine  SALIANT  for  the  benefit  team  SA 
researchers,  practitioners,  and,  more  importantly,  teams  who  rely  on  this  skill  for  the  effective  and  safe  accomplishment 
of  their  missions. 


ACKNOWLEDGMENTS 

The  opinions  expressed  here  are  those  of  the  authors  and  do  not  constitute  an  official  position  of  their 
affiliated  organizations.  We  would  like  to  thank  Randall  L.  Oser  for  comments  on  earlier  draffs  of  this  paper. 

REFERENCES 

Arbak,  C.  J.,  Schwartz,  N.,  &  Kuperman,  G.  (1987).  Evaluating  the  panoramic  cockpit  controls  and  displays 
system.  Paper  presented  at  the  Fourth  Annual  Symposium  on  Aviation  Psychology,  Columbus,  Ohio. 

Blickensderfer,  E.  L.  (1996).  Training  teams  to  self-correct:  Effect  of  intra-team  feedback  on  shared 
understanding  and  coordinated  task  accomplishment.  Unpublished  Master’s  thesis.  University  of  Central  Florida, 
Orlando,  Florida. 


188 


Bowers,  C.  A.,  Deaton,  J.,  Oser,  R.,  Prince,  C.,  &,  Kolb,  M.  (1995)  Impact  of  automation  on  aircrew 
communication  and  decision  making  performance.  The  International  Journal  of  Aviation  Psychology,  5,  145-168. 

Bowers,  C.  A.,  Braun,  C.,  &  Kline,  P.  B.  (1994).  Communication  and  team  situational  awareness.  In  R 
D.  Gilson,  T  J.  Garland,  &  J.  M.  Koonce,  Situational  awarenec  in  complex  systems  (pp.  305-314).  Daytona 
Beach,  FL:  Embry-Riddle  Aeronautical  University  Press. 

Bowers,  C.  A.,  Barnett,  J.,  Weaver,  J.,  &  Stout,  R.  J.  (1998).  Empirical  validation  of  the  SAL1ANT 
methodology.  The  First  Annual  Symposium/Business  Meeting  of  the  Human  Factors  &  Medicine  Panel  on 
Collaborative  Crew  Performance  in  Complex  Systems,  Edinburg,  United  Kingdom. 

Brannick,  M.  T,  Roach,  R,  &  Salas,  E.  (1993).  Understanding  team  performance.  A  multimethod  study. 
Human  Performance,  6,  287-308 

Bunecke,  J.  L.,  Povenmire,  11.  K.,  Rockway,  M.  R.  &  Patton,  M.  W.  (1990).  The  awareness  component  of 
cockpit  resource  management.  Proceedings  of  Psychology  in  the  Department  of  Defense,  (pp.  259-263).  USAF 
Academy  Department  of  Behavioral  Sciences  and  Leadership 

Endsley,  M  R.  (1988).  Design  and  evaluation  for  situation  awareness  enhancement  Proceedings  of  the 
Human  Factors  Society  32nd  Annual  Meeting,  97-101. 

Endsley,  M.  R.  (1995).  Toward  a  theory  of  situation  awareness  in  dynamic  systems.  In  R.  D.  Gilson 
(Ed.),  Human  Factors,  Special  Issue:  Situation  Awareness,  37,  65-84. 

Foushee,  H.  C.  (1984).  Dyads  and  triads  at  35,  000  feet’  Factors  affecting  group  process  and  aircrew 
performance,  American  Psychologist,  39,  885-893. 

Fowlkes,  J  E.,  Lane,  N„  E.,  Salas,  E.,  Franz,  T.,  &  Oser,  R.  L.  (1994).  Improving  the  measurement  of  team 
performance:  The  TARGETS  methodology.  Military  Psychology,  6, 47-61 

Fracker,  M  L.  (1991).  Measures  of  situation  awareness:  Review  and  future  directions  (Rep.  No.  AL-TR- 
1991-0128).  Wright-Patterson  Air  Force,  Ohio:  Armstrong  Laboratories. 

Hartel,  C.  E.  J.,  Smith,  K.,  &  Prince,  C.  (1991).  Defining  aircrew  coordination:  Searching  for  mishaps  for 
meaning.  Paper  presented  at  the  Sixth  International  Symposium  on  Aviation  Psychology,  Columbus,  OH 

Lawlis,  G.  F.,  &  Lu,  E.  (1972).  Judgment  of  counseling  process:  Reliability,  agreement,  and  error. 
Psychological  Bulletin,  78,  17-20. 

Leedom,  D  K.  (1990).  Aircrew  coordination  training-  A  new  approach.  Fort  Rucker,  Alabama:  U.S.  Army 
Research  Institute  Aviation  R&D  activity. 

Mosier,  K.  L.  &  Chidester,  T.  R.  (1991).  Situation  assessment  and  situation  awareness  in  a  team  setting. 
Proceedings  of  the  1 1th  Congress  of  the  International  Ergonomics  Association,  Paris. 

Mufiiz,  E.,  Stout,  R.  J,,  &  Salas,  E.  (1996,  March).  Communication  as  an  indicator  of  team  situation 
awareness.  Paper  presented  at  the  42nd  Annual  Meeting  of  the  Southeastern  Psychological  Association,  Norfolk,  VA. 

Mufiiz,  E.  J.,  Stout,  R.  J.,  Bowers,  C.  A.,  &  Salas,  E.  (1998).  A  methodology  for  measuring  team  situational 
awareness:  Situational  Linked  Indicators  Adapted  to  Novel  Tasks  (SAL1ANT).  The  First  Annual 

Symposium/Business  Meeting  of  the  Human  Factors  &  Medicine  Panel  on  Collaborative  Crew  Performance  in 
Complex  Systems,  Edinburg,  United  Kingdom. 

Oser,  R.  L.,  Prince,  C,,  Morgan,  B.  B.,  Jr.,  &  Simpson,  S.  S.  (1991).  An  analysis  of  aircrew  communication 
patterns  and  content  (Technical  Report  90-009).  Orlando,  FL:  Nava!  Training  Systems  Center 

Palmer,  E.  (1990).  Crew  situation  awareness.  Proceedings  of  the  Human  Factors  Society  34th  Annual 
Meeting,  22. 

Prince,  C.  (1998)  Guidelines  for  situation  awareness  training.  (Preliminary  report  by  NAWCTSD,  UCF,  & 
FAA).  Orlando,  FL. 

Prince,  C.,  &  Salas,  E.  (1993).  Training  and  research  for  teamwork  in  the  military'  aircrew.  In  E.  L.  Wiener, 
B.  G.  Kanki,  &  R.  L.  Helmreich  (Eds.),  Cockpit  resource  management  (pp.  337-366).  Orlando,  FL:  Academic  Press. 

Salas,  E,,  Prince,  C.,  Baker,  D.  P.,  &  Shrestha,  L.  (1995).  Situation  awareness  in  team  performance: 
Implications  for  measurement.  In  R.  D.  Gilson  (Ed.),  Human  Factors.  Special  Issue:  Situation  Awareness,  37,  123- 
136. 

Schwartz,  D.  (1990).  Training  for  situational  awareness.  Houston,  TX:  Flight  Safety  International. 

Smith-Jentsh,  K.  A.,  Salas,  E.,  &  Baker,  D.  (1996).  Training  team  performance-related  assertiveness. 
Personnel  Psychology,  49,  909-936. 

Stout,  R.  J.  (1995).  Planning  effects  on  communication  strategies:  A  shared  mental  model  perspective 
Proceedings  of  the  Human  Factors  &  Ergonomics  Society  39th  Annual  Meeting,  Santa  Monica,  CA,  1278-1282. 


189 


Stout,  R.  J.,  Cannon-Bowers,  J.  A.,  &  Salas,  E.  (1996).  The  role  of  shared  mental  models  in  developing  team 
situational  awareness:  Implications  for  training.  Training  Research  Journal,  2,  85-1 16. 

Wagner,  M.,  &  Simon,  R.  (1990,  August).  Development  of  measures  of  crew  coordination.  (Contract 
DAHC3j-89-D-0030). 

Wellens,  A.  R.  (1993).  Group  situation  awareness  and  distributed  decision  making  from  military  to  civilian 
applications.  In  N.  J.  Castellan,  Jr.  (Ed.),  Individual  and  group  decision  making:  Current  issues  (pp.  267-291). 


190 


